batch_size_fail_threshold_in_kb (integer, Minimum: 1, Maximum: 1000000). Fail any multiple-partition batch exceeding this value. 50kb (10x warn threshold) by default.
batch_size_warn_threshold_in_kb (integer, Minimum: 1, Maximum: 1000000). Log a warning message on any multiple-partition batch size exceeding this value.5kb per batch by default.Caution should be taken on increasing the size of this thresholdas it can lead to node instability.datacenter (string, MaxLength: 128). Name of the datacenter to which nodes of this service belong. Can be set only when creating the service.read_request_timeout_in_ms (integer, Minimum: 1000, Maximum: 10000). How long the coordinator waits for read operations to complete before timing it out. 5 seconds by default.write_request_timeout_in_ms (integer, Minimum: 1000, Maximum: 10000). How long the coordinator waits for write requests to complete with at least one node in the local datacenter. 2 seconds by default.Appears on spec.userConfig.
Allow incoming connections from CIDR address block, e.g. 10.20.0.0/16.
CIDR address block, either as a string, or in a dict with an optional description field.
Required
network (string, MaxLength: 43). CIDR address block.Appears on spec.userConfig.
Allow incoming connections from CIDR address block, e.g. 10.20.0.0/16.
CIDR address block, either as a string, or in a dict with an optional description field.
Required
network (string, MaxLength: 43). CIDR address block.Optional
authSecretRef (object). Authentication reference to Aiven token in a secret. See below for nested schema.databaseName (string, Immutable, MaxLength: 63). Specifies the Clickhouse database name. Defaults to metadata.name if omitted.
-Note: metadata.name is ASCII-only. For UTF-8 names, use spec.databaseName, but ASCII is advised for compatibility.databaseName (string, Immutable, MaxLength: 63). Specifies the Clickhouse database name. Defaults to metadata.name if omitted.Note
+metadata.name is ASCII-only. For UTF-8 names, use spec.databaseName, but ASCII is advised for compatibility.
Appears on spec.
Authentication reference to Aiven token in a secret.
diff --git a/api-reference/clickhousegrant.html b/api-reference/clickhousegrant.html index dbde5bad..96483c1d 100644 --- a/api-reference/clickhousegrant.html +++ b/api-reference/clickhousegrant.html @@ -2034,8 +2034,8 @@ClickhouseGrant is the Schema for the ClickhouseGrants API
Ambiguity in the GRANT syntax
Due to an ambiguity in the GRANT syntax in Clickhouse, you should not have users and roles with the same name. It is not clear if a grant refers to the user or the role.
Warning
+Due to the way ClickHouse operates, updating this resource first revokes the existing privileges.
Required
Appears on spec.
Configuration to grant a privilege. Privileges not in the manifest are revoked. Existing privileges are retained; new ones are granted.
+PrivilegeGrant represents the privileges to be granted to users or roles. +See.
Required
database (string). The database that the grant refers to.grantees (array of objects, MinItems: 1). List of grantees (users or roles) to grant the privilege to. See below for nested schema.privileges (array of strings). The privileges to grant, i.e. INSERT, SELECT.
-See https://clickhouse.com/docs/en/sql-reference/statements/grant#assigning-role-syntax.Optional
table (string). The tables that the grant refers to. To grant a privilege on all tables in a database, omit this field instead of writing table: "*".
withGrantOption (boolean). If true, then the grantee (user or role) get the permission to execute the GRANT query.
Users can grant privileges of the same scope they have and less.
-See https://clickhouse.com/docs/en/sql-reference/statements/grant#granting-privilege-syntax.Appears on spec.privilegeGrants.
List of grantees (users or roles) to grant the privilege to.
+Grantee represents a user or a role to which privileges or roles are granted.
+Ambiguity in the GRANT syntax
Due to an ambiguity in the GRANT syntax in ClickHouse, +users and roles should not share the same name. +It is unclear whether a grant applies to the user or the role.
+Optional
role (string).Appears on spec.
Configuration to grant a role. Role grants not in the manifest are revoked. Existing role grants are retained; new ones are granted.
+RoleGrant represents the roles to be assigned to users or roles. +See.
Required
grantees (array of objects, MinItems: 1). List of grantees (users or roles) to grant the privilege to. See below for nested schema.withAdminOption (boolean). If true, the grant is executed with ADMIN OPTION privilege.
-See https://clickhouse.com/docs/en/sql-reference/statements/grant#admin-option.Appears on spec.roleGrants.
List of grantees (users or roles) to grant the privilege to.
+Grantee represents a user or a role to which privileges or roles are granted.
+Ambiguity in the GRANT syntax
Due to an ambiguity in the GRANT syntax in ClickHouse, +users and roles should not share the same name. +It is unclear whether a grant applies to the user or the role.
+Optional
role (string).authSecretRef (object). Authentication reference to Aiven token in a secret. See below for nested schema.connInfoSecretTarget (object). Secret configuration. See below for nested schema.connInfoSecretTargetDisabled (boolean, Immutable). When true, the secret containing connection information will not be created, defaults to false. This field cannot be changed after resource creation.username (string, Immutable, MaxLength: 63). Name of the Clickhouse user. Defaults to metadata.name if omitted.
-Note: metadata.name is ASCII-only. For UTF-8 names, use spec.username, but ASCII is advised for compatibility.username (string, Immutable, MaxLength: 63). Name of the Clickhouse user. Defaults to metadata.name if omitted.Note
+metadata.name is ASCII-only. For UTF-8 names, use spec.username, but ASCII is advised for compatibility.
Appears on spec.
Authentication reference to Aiven token in a secret.
diff --git a/api-reference/grafana.html b/api-reference/grafana.html index a364033f..46459071 100644 --- a/api-reference/grafana.html +++ b/api-reference/grafana.html @@ -2416,7 +2416,7 @@Appears on spec.userConfig.
Allow incoming connections from CIDR address block, e.g. 10.20.0.0/16.
CIDR address block, either as a string, or in a dict with an optional description field.
Required
network (string, MaxLength: 43). CIDR address block.Appears on spec.userConfig.
Allow incoming connections from CIDR address block, e.g. 10.20.0.0/16.
CIDR address block, either as a string, or in a dict with an optional description field.
Required
network (string, MaxLength: 43). CIDR address block.Appears on spec.userConfig.
Configure external secret providers in order to reference external secrets in connector configuration. Currently Hashicorp Vault (provider: vault, auth_method: token) and AWS Secrets Manager (provider: aws, auth_method: credentials) are supported. Secrets can be referenced in connector config with ${
SecretProvider.
Required
name (string). Name of the secret provider. Used to reference secrets in connector config.Appears on spec.userConfig.
Allow incoming connections from CIDR address block, e.g. 10.20.0.0/16.
CIDR address block, either as a string, or in a dict with an optional description field.
Required
network (string, MaxLength: 43). CIDR address block.Appears on spec.userConfig.
Configure external secret providers in order to reference external secrets in connector configuration. Currently Hashicorp Vault (provider: vault, auth_method: token) and AWS Secrets Manager (provider: aws, auth_method: credentials) are supported. Secrets can be referenced in connector config with ${
SecretProvider.
Required
name (string). Name of the secret provider. Used to reference secrets in connector config.Appears on spec.userConfig.
Allow incoming connections from CIDR address block, e.g. 10.20.0.0/16.
CIDR address block, either as a string, or in a dict with an optional description field.
Required
network (string, MaxLength: 43). CIDR address block.Appears on spec.userConfig.
Index patterns.
+Allows you to create glob style patterns and set a max number of indexes matching this pattern you want to keep. Creating indexes exceeding this value will cause the oldest one to get deleted. You could for example create a pattern looking like logs.? and then create index logs.1, logs.2 etc, it will delete logs.1 once you create logs.6. Do note logs.? does not apply to logs.10. Note: Setting max_index_count to 0 will do nothing and the pattern gets ignored.
Required
max_index_count (integer, Minimum: 0). Maximum number of indexes to keep.Appears on spec.userConfig.
Allow incoming connections from CIDR address block, e.g. 10.20.0.0/16.
CIDR address block, either as a string, or in a dict with an optional description field.
Required
network (string, MaxLength: 43). CIDR address block.ism_history_max_docs (integer, Minimum: 1). The maximum number of documents before rolling over the audit history index.
ism_history_rollover_check_period (integer, Minimum: 1, Maximum: 2147483647). The time between rollover checks for the audit history index in hours.ism_history_rollover_retention_period (integer, Minimum: 1, Maximum: 2147483647). How long audit history indices are kept in days.knn_memory_circuit_breaker_enabled (boolean). Enable or disable KNN memory circuit breaker. Defaults to true.knn_memory_circuit_breaker_limit (integer, Minimum: 3, Maximum: 100). Maximum amount of memory that can be used for KNN index. Defaults to 50% of the JVM heap size.override_main_response_version (boolean). Compatibility mode sets OpenSearch to report its version as 7.10 so clients continue to work. Default is false.plugins_alerting_filter_by_backend_roles (boolean). Enable or disable filtering of alerting by backend roles. Requires Security plugin. Defaults to false.reindex_remote_whitelist (array of strings, MaxItems: 32). Whitelisted addresses for reindexing. Changing this value will cause all OpenSearch instances to restart.Appears on spec.userConfig.
Allow incoming connections from CIDR address block, e.g. 10.20.0.0/16.
CIDR address block, either as a string, or in a dict with an optional description field.
Required
network (string, MaxLength: 43). CIDR address block.jit (boolean). Controls system-wide use of Just-in-Time Compilation (JIT).
log_autovacuum_min_duration (integer, Minimum: -1, Maximum: 2147483647). Causes each action executed by autovacuum to be logged if it ran for at least the specified number of milliseconds. Setting this to zero logs all autovacuum actions. Minus-one (the default) disables logging autovacuum actions.log_error_verbosity (string, Enum: TERSE, DEFAULT, VERBOSE). Controls the amount of detail written in the server log for each message that is logged.log_line_prefix (string, Enum: 'pid=%p,user=%u,db=%d,app=%a,client=%h ', '%t [%p]: [%l-1] user=%u,db=%d,app=%a,client=%h ', '%m [%p] %q[user=%u,db=%d,app=%a] '). Choose from one of the available log formats.log_line_prefix (string, Enum: 'pid=%p,user=%u,db=%d,app=%a,client=%h ', '%t [%p]: [%l-1] user=%u,db=%d,app=%a,client=%h ', '%m [%p] %q[user=%u,db=%d,app=%a] ', 'pid=%p,user=%u,db=%d,app=%a,client=%h,txid=%x,qid=%Q '). Choose from one of the available log formats.log_min_duration_statement (integer, Minimum: -1, Maximum: 86400000). Log statements that take more than this number of milliseconds to run, -1 disables.log_temp_files (integer, Minimum: -1, Maximum: 2147483647). Log statements for each temporary file created larger than this number of kilobytes, -1 disables.max_files_per_process (integer, Minimum: 1000, Maximum: 4096). PostgreSQL maximum number of files that can be open per process.Appears on spec.userConfig.
Allow incoming connections from CIDR address block, e.g. 10.20.0.0/16.
CIDR address block, either as a string, or in a dict with an optional description field.
Required
network (string, MaxLength: 43). CIDR address block.Appears on spec.clickhouseKafka.
Tables to create.
+Table to create.
Required
columns (array of objects, MaxItems: 100). Table columns. See below for nested schema.Appears on spec.clickhouseKafka.tables.
Table columns.
+Table column.
Required
name (string, MinLength: 1, MaxLength: 40). Column name.Appears on spec.clickhouseKafka.tables.
Kafka topics.
+Kafka topic.
Required
name (string, MinLength: 1, MaxLength: 249). Name of the topic.Appears on spec.clickhousePostgresql.
Databases to expose.
+Database to expose.
Optional
database (string, MinLength: 1, MaxLength: 63). PostgreSQL database to expose.Appears on spec.datadog.
Custom tags provided by user.
+Datadog tag defined by user.
Required
tag (string, MinLength: 1, MaxLength: 200). Tag format and usage are described here: https://docs.datadoghq.com/getting_started/tagging. Tags with prefix aiven- are reserved for Aiven.Appears on spec.externalAWSCloudwatchMetrics.
Metrics to not send to AWS CloudWatch (takes precedence over extra_metrics).
+Metric name and subfield.
Required
field (string, MaxLength: 1000). Identifier of a value in the metric.Appears on spec.externalAWSCloudwatchMetrics.
Metrics to allow through to AWS CloudWatch (in addition to default metrics).
+Metric name and subfield.
Required
field (string, MaxLength: 1000). Identifier of a value in the metric.Appears on spec.datadog.
Custom tags provided by user.
+Datadog tag defined by user.
Required
tag (string, MinLength: 1, MaxLength: 200). Tag format and usage are described here: https://docs.datadoghq.com/getting_started/tagging. Tags with prefix aiven- are reserved for Aiven.PostgreSQL: wait for a valid backup to create read replicaClickhouseGrant: grant privileges for an unknown table (Clickhouse can do that)ClickhouseGrant: track the state to revoke only known privilegesCassandra field userConfig.cassandra.read_request_timeout_in_ms, type integer: How long the
+ coordinator waits for read operations to complete before timing it outCassandra field userConfig.cassandra.write_request_timeout_in_ms, type integer: How long
+ the coordinator waits for write requests to complete with at least one node in the local datacenterOpenSearch field userConfig.opensearch.knn_memory_circuit_breaker_enabled, type boolean:
+ Enable or disable KNN memory circuit breaker. Defaults to trueOpenSearch field userConfig.opensearch.knn_memory_circuit_breaker_limit, type integer: Maximum
+ amount of memory that can be used for KNN index. Defaults to 50% of the JVM heap sizePostgreSQL field userConfig.pg.log_line_prefix: enum ['%m [%p] %q[user=%u,db=%d,app=%a]
+', '%t [%p]: [%l-1] user=%u,db=%d,app=%a,client=%h ', 'pid=%p,user=%u,db=%d,app=%a,client=%h ']['%m [%p] %q[user=%u,db=%d,app=%a] ', '%t [%p]: [%l-1] user=%u,db=%d,app=%a,client=%h ',
+'pid=%p,user=%u,db=%d,app=%a,client=%h ', 'pid=%p,user=%u,db=%d,app=%a,client=%h,txid=%x,qid=%Q ']http.StatusBadRequest on ClickhouseGrant deletionProvision and manage Aiven services from your Kubernetes cluster.
"},{"location":"index.html#what-is-aiven","title":"What is Aiven?","text":"Aiven offers managed services for the best open source data technologies, on a cloud of your choice.
We offer multiple cloud options because we believe that everyone should have access to great data platforms wherever they host their applications. Our customers tell us they love it because they know that they aren\u2019t locked in to one particular cloud platform for all their data needs.
"},{"location":"index.html#contributing","title":"Contributing","text":"The contribution guide covers everything you need to know about how you can contribute to Aiven Operator for Kubernetes. The developer guide will help you onboard as a developer.
"},{"location":"authentication.html","title":"Authenticating","text":"To get authenticated and authorized, set up the communication between the Aiven Operator for Kubernetes and Aiven by using a token stored in a Kubernetes Secret. You can then refer to the Secret name on every custom resource in the authSecretRef field.
Important
If you don't have an Aiven account yet, sign up here for a free trial \ud83e\udd80
1. Generate an authentication token
Refer to our documentation article to generate your authentication token.
2. Create the Kubernetes Secret
The following command creates a Secret named aiven-token with a token field containing the authentication token:
kubectl create secret generic aiven-token --from-literal=token=\"TOKEN\"\nWhen managing your Aiven resources, we will be using the created Secret in the authSecretRef field. It will look like the example below:
apiVersion: aiven.io/v1alpha1\nkind: PostgreSQL\nmetadata:\n name: pg-sample\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n [ ... ]\nAlso, note that within Aiven, all resources are conceptually inside a Project. By default, a random project name is generated when you signup, but you can also create new projects.
The Project name is required in most of the resources. It will look like the example below:
apiVersion: aiven.io/v1alpha1\nkind: PostgreSQL\nmetadata:\n name: pg-sample\nspec:\n project: PROJECT_NAME\n [ ... ]\nhttp.StatusBadRequest on ClickhouseGrant deletionClickhouseGrant invalid remote and local privileges comparisonClickhouseGrant: doesn't escape role name to grantClickhouseUser: password was reset due to an incorrect processing cycleClickhouseRole deletion error (missing database)ClickhouseGrant deletion errors (missing database, service, role)REBALANCING stateClickhouseGrantKafkaConnect field userConfig.secret_providers, type array: Configure external secret providers in order to reference external secrets in connector configurationKafka field userConfig.kafka_connect_secret_providers, type array: Configure external secret providers in order to reference external secrets in connector configurationKafka field userConfig.letsencrypt_sasl_privatelink, type boolean: Use Letsencrypt CA for Kafka SASL via PrivatelinkServiceIntegration field datadog.mirrormaker_custom_metrics, type array: List of custom metricsServiceIntegration field kafkaMirrormaker.kafka_mirrormaker.consumer_auto_offset_reset, type string: Set where consumer starts to consume dataServiceIntegration field kafkaMirrormaker.kafka_mirrormaker.consumer_max_poll_records, type integer: Set consumer max.poll.records. The default is 500PostgreSQL field userConfig.pgaudit: deprecatedServiceIntegrationEndpoint field externalPostgresql.ssl_mode: enum [allow, disable, prefer, require, verify-ca, verify-full] \u2192 [require, verify-ca, verify-full]ServiceIntegrationEndpointServiceIntegration flink_external_postgresql typeServiceIntegration field datadog.datadog_pgbouncer_enabled, type boolean: Enable Datadog PgBouncer Metric TrackingServiceIntegration deletion when instance has no id setdisk_space field validationproject, serviceName fields validationConnectionPool doesn't check service user preconditionCA_CERT secret key for Grafana, OpenSearch, Redis, and Clickhouse. Can't be used with these service types ddog-gov.com, us3.datadoghq.com, us5.datadoghq.com]`ServiceIntegrationEndpoint field externalKafka.ssl_endpoint_identification_algorithm: enum [, https] \u2192 [https]ClickhouseUser webhook. Doesn't do any validation or mutationKafka field userConfig.kafka_version: enum [3.4, 3.5, 3.6] \u2192 [3.4, 3.5, 3.6, 3.7]ServiceIntegrationEndpoint field datadog.site: enum [datadoghq.com, datadoghq.eu, ddog-gov.com, us3.datadoghq.com, us5.datadoghq.com] \u2192 `[ap1.datadoghq.com, datadoghq.com, datadoghq.eu,ClickhouseRoleKafkaACLKafkaSchemaRegistryACL kindClickhouseDatabase kindDatabase kind validations and default values with CRD validation rulesREDIS_CA_CERT, MYSQL_CA_CERT, etc.KafkaTopic field config.local_retention_bytes, type integer: local.retention.bytes valueKafkaTopic field config.local_retention_ms, type integer: local.retention.ms valueKafkaTopic field config.remote_storage_enable, type boolean: remote_storage_enableCassandra field userConfig.cassandra_version: pattern ^[0-9]+(\\.[0-9]+)?$Cassandra field userConfig.project_to_fork_from: pattern ^[a-z][-a-z0-9]{0,63}$|^$Cassandra field userConfig.service_to_fork_from: pattern ^[a-z][-a-z0-9]{0,63}$|^$Cassandra field userConfig.service_to_join_with: pattern ^[a-z][-a-z0-9]{0,63}$Clickhouse field userConfig.project_to_fork_from: pattern ^[a-z][-a-z0-9]{0,63}$|^$Clickhouse field userConfig.service_to_fork_from: pattern ^[a-z][-a-z0-9]{0,63}$|^$Grafana field userConfig.project_to_fork_from: pattern ^[a-z][-a-z0-9]{0,63}$|^$Grafana field userConfig.service_to_fork_from: pattern ^[a-z][-a-z0-9]{0,63}$|^$Kafka field userConfig.kafka.sasl_oauthbearer_expected_audience: pattern ^[^\\r\\n]*$Kafka field userConfig.kafka.sasl_oauthbearer_expected_issuer: pattern ^[^\\r\\n]*$Kafka field userConfig.kafka.sasl_oauthbearer_sub_claim_name: pattern ^[^\\r\\n]*$MySQL field userConfig.mysql.default_time_zone: pattern ^([-+][\\d:]*|[\\w/]*)$MySQL field userConfig.project_to_fork_from: pattern ^[a-z][-a-z0-9]{0,63}$|^$MySQL field userConfig.service_to_fork_from: pattern ^[a-z][-a-z0-9]{0,63}$|^$OpenSearch field userConfig.openid.client_id: pattern ^[^\\r\\n]*$OpenSearch field userConfig.openid.client_secret: pattern ^[^\\r\\n]*$OpenSearch field userConfig.openid.header: pattern ^[^\\r\\n]*$OpenSearch field userConfig.openid.jwt_header: pattern ^[^\\r\\n]*$OpenSearch field userConfig.openid.jwt_url_parameter: pattern ^[^\\r\\n]*$OpenSearch field userConfig.openid.roles_key: pattern ^[^\\r\\n]*$OpenSearch field userConfig.openid.scope: pattern ^[^\\r\\n]*$OpenSearch field userConfig.openid.subject_key: pattern ^[^\\r\\n]*$OpenSearch field userConfig.project_to_fork_from: pattern ^[a-z][-a-z0-9]{0,63}$|^$OpenSearch field userConfig.saml.idp_entity_id: pattern ^[^\\r\\n]*$OpenSearch field userConfig.saml.roles_key: pattern ^[^\\r\\n]*$OpenSearch field userConfig.saml.sp_entity_id: pattern ^[^\\r\\n]*$OpenSearch field userConfig.saml.subject_key: pattern ^[^\\r\\n]*$OpenSearch field userConfig.service_to_fork_from: pattern ^[a-z][-a-z0-9]{0,63}$|^$PostgreSQL field userConfig.pg.timezone: pattern ^[\\w/]*$PostgreSQL field userConfig.pg_service_to_fork_from: pattern ^[a-z][-a-z0-9]{0,63}$|^$PostgreSQL field userConfig.project_to_fork_from: pattern ^[a-z][-a-z0-9]{0,63}$|^$PostgreSQL field userConfig.service_to_fork_from: pattern ^[a-z][-a-z0-9]{0,63}$|^$Redis field userConfig.project_to_fork_from: pattern ^[a-z][-a-z0-9]{0,63}$|^$Redis field userConfig.service_to_fork_from: pattern ^[a-z][-a-z0-9]{0,63}$|^$OpenSearch field userConfig.opensearch.plugins_alerting_filter_by_backend_roles, type boolean: Enable or disable filtering of alerting by backend roles. Requires Security pluginRedis field userConfig.redis_notify_keyspace_events: pattern ^[KEg\\$lshzxeA]*$ \u2192 ^[KEg\\$lshzxentdmA]*$PostgreSQL field userConfig.pgaudit, type object: System-wide settings for the pgaudit extensionServiceIntegration field datadog.opensearch.cluster_stats_enabled, type boolean: Enable Datadog Opensearch Cluster MonitoringOpenSearch field userConfig.opensearch.enable_security_audit, type boolean: Enable/Disable security auditKafka field userConfig.kafka_rest_config.name_strategy, type string: Name strategy to use when selecting subject for storing schemasRedis field userConfig.redis_version, type string: Redis major versionGrafana field userConfig.auth_github.auto_login, type boolean: Allow users to bypass the login screen and automatically log inGrafana field userConfig.auth_github.skip_org_role_sync, type boolean: Stop automatically syncing user rolesClickhouse field userConfig.additional_backup_regions: deprecatedGrafana field userConfig.additional_backup_regions: deprecatedKafkaConnect field userConfig.additional_backup_regions: deprecatedKafka field userConfig.additional_backup_regions: deprecatedOpenSearch field userConfig.additional_backup_regions: deprecatedRedis field userConfig.additional_backup_regions: deprecatedCassandra field userConfig.cassandra_version: enum [3, 4, 4.1] \u2192 [4, 4.1]Kafka field userConfig.kafka_version: enum [3.1, 3.3, 3.4, 3.5, 3.6] \u2192 [3.4, 3.5, 3.6]PostgreSQL field userConfig.pg_version: enum [11, 12, 13, 14, 15, 16] \u2192 [12, 13, 14, 15, 16]Cassandra field technicalEmails, type array: Defines the email addresses that will receive alerts about upcoming maintenance updates or warnings about service instabilityClickhouse field technicalEmails, type array: Defines the email addresses that will receive alerts about upcoming maintenance updates or warnings about service instabilityGrafana field technicalEmails, type array: Defines the email addresses that will receive alerts about upcoming maintenance updates or warnings about service instabilityKafkaConnect field technicalEmails, type array: Defines the email addresses that will receive alerts about upcoming maintenance updates or warnings about service instabilityKafka field technicalEmails, type array: Defines the email addresses that will receive alerts about upcoming maintenance updates or warnings about service instabilityMySQL field technicalEmails, type array: Defines the email addresses that will receive alerts about upcoming maintenance updates or warnings about service instabilityOpenSearch field technicalEmails, type array: Defines the email addresses that will receive alerts about upcoming maintenance updates or warnings about service instabilityPostgreSQL field technicalEmails, type array: Defines the email addresses that will receive alerts about upcoming maintenance updates or warnings about service instabilityRedis field technicalEmails, type array: Defines the email addresses that will receive alerts about upcoming maintenance updates or warnings about service instabilityCassandra field connInfoSecretTargetDisabled, type boolean: When true, the secret containing connection information will not be created, defaults to falseClickhouse field connInfoSecretTargetDisabled, type boolean: When true, the secret containing connection information will not be created, defaults to falseClickhouseUser field connInfoSecretTargetDisabled, type boolean: When true, the secret containing connection information will not be created, defaults to falseConnectionPool field connInfoSecretTargetDisabled, type boolean: When true, the secret containing connection information will not be created, defaults to falseGrafana field connInfoSecretTargetDisabled, type boolean: When true, the secret containing connection information will not be created, defaults to falseKafka field connInfoSecretTargetDisabled, type boolean: When true, the secret containing connection information will not be created, defaults to falseMySQL field connInfoSecretTargetDisabled, type boolean: When true, the secret containing connection information will not be created, defaults to falseOpenSearch field connInfoSecretTargetDisabled, type boolean: When true, the secret containing connection information will not be created, defaults to falsePostgreSQL field connInfoSecretTargetDisabled, type boolean: When true, the secret containing connection information will not be created, defaults to falseProject field connInfoSecretTargetDisabled, type boolean: When true, the secret containing connection information will not be created, defaults to falseRedis field connInfoSecretTargetDisabled, type boolean: When true, the secret containing connection information will not be created, defaults to falseServiceUser field connInfoSecretTargetDisabled, type boolean: When true, the secret containing connection information will not be created, defaults to falseKAFKA_SCHEMA_REGISTRY_URI and KAFKA_REST_URI to Kafka secretCONNECTIONPOOL_NAME in ConnectionPool secretCONNECTIONPOOL_PORT exposes service port instead of pool portSERVICEUSER_PORT when sasl is the only authentication methodPostgreSQL field userConfig.pg_qualstats.enabled: deprecatedPostgreSQL field userConfig.pg_qualstats.min_err_estimate_num: deprecatedPostgreSQL field userConfig.pg_qualstats.min_err_estimate_ratio: deprecatedPostgreSQL field userConfig.pg_qualstats.track_constants: deprecatedPostgreSQL field userConfig.pg_qualstats.track_pg_catalog: deprecatedPreconditions, CreateOrUpdate, Delete. Thanks to @ataraxKafka field userConfig.kafka.transaction_partition_verification_enable, type boolean: Enable verification that checks that the partition has been added to the transaction before writing transactional records to the partitionCassandra field userConfig.service_log, type boolean: Store logs for the service so that they are available in the HTTP API and consoleClickhouse field userConfig.service_log, type boolean: Store logs for the service so that they are available in the HTTP API and consoleGrafana field userConfig.service_log, type boolean: Store logs for the service so that they are available in the HTTP API and consoleKafkaConnect field userConfig.service_log, type boolean: Store logs for the service so that they are available in the HTTP API and consoleKafka field userConfig.kafka_rest_config.name_strategy_validation, type boolean: If true, validate that given schema is registered under expected subject name by the used name strategy when producing messagesKafka field userConfig.service_log, type boolean: Store logs for the service so that they are available in the HTTP API and consoleMySQL field userConfig.service_log, type boolean: Store logs for the service so that they are available in the HTTP API and consoleOpenSearch field userConfig.service_log, type boolean: Store logs for the service so that they are available in the HTTP API and consolePostgreSQL field userConfig.pg_qualstats, type object: System-wide settings for the pg_qualstats extensionPostgreSQL field userConfig.service_log, type boolean: Store logs for the service so that they are available in the HTTP API and consoleRedis field userConfig.service_log, type boolean: Store logs for the service so that they are available in the HTTP API and consoleServiceIntegration: do not send empty user config to the APIstring type fields to the documentationClickhouse field userConfig.private_access.clickhouse_mysql, type boolean: Allow clients to connect to clickhouse_mysql with a DNS name that always resolves to the service's private IP addressesClickhouse field userConfig.privatelink_access.clickhouse_mysql, type boolean: Enable clickhouse_mysqlClickhouse field userConfig.public_access.clickhouse_mysql, type boolean: Allow clients to connect to clickhouse_mysql from the public internet for service nodes that are in a project VPC or another type of private networkGrafana field userConfig.unified_alerting_enabled, type boolean: Enable or disable Grafana unified alerting functionalityKafka field userConfig.aiven_kafka_topic_messages, type boolean: Allow access to read Kafka topic messages in the Aiven Console and REST APIKafka field userConfig.kafka.sasl_oauthbearer_expected_audience, type string: The (optional) comma-delimited setting for the broker to use to verify that the JWT was issued for one of the expected audiencesKafka field userConfig.kafka.sasl_oauthbearer_expected_issuer, type string: Optional setting for the broker to use to verify that the JWT was created by the expected issuerKafka field userConfig.kafka.sasl_oauthbearer_jwks_endpoint_url, type string: OIDC JWKS endpoint URL. By setting this the SASL SSL OAuth2/OIDC authentication is enabledKafka field userConfig.kafka.sasl_oauthbearer_sub_claim_name, type string: Name of the scope from which to extract the subject claim from the JWT. Defaults to subKafka field userConfig.kafka_version: enum [3.1, 3.3, 3.4, 3.5] \u2192 [3.1, 3.3, 3.4, 3.5, 3.6]Kafka field userConfig.tiered_storage.local_cache.size: deprecatedOpenSearch field userConfig.opensearch.indices_memory_max_index_buffer_size, type integer: Absolute value. Default is unbound. Doesn't work without indices.memory.index_buffer_sizeOpenSearch field userConfig.opensearch.indices_memory_min_index_buffer_size, type integer: Absolute value. Default is 48mb. Doesn't work without indices.memory.index_buffer_sizeOpenSearch field userConfig.opensearch.auth_failure_listeners.internal_authentication_backend_limiting.authentication_backend: enum [internal]OpenSearch field userConfig.opensearch.auth_failure_listeners.internal_authentication_backend_limiting.type: enum [username]OpenSearch field userConfig.opensearch.auth_failure_listeners.ip_rate_limiting.type: enum [ip]OpenSearch field userConfig.opensearch.search_max_buckets: maximum 65536 \u2192 1000000ServiceIntegration field kafkaMirrormaker.kafka_mirrormaker.producer_max_request_size: maximum 67108864 \u2192 268435456projectVpcId and projectVPCRef mutablenil user config conversionCassandra kind option additional_backup_regionsGrafana kind option auto_loginKafka kind properties log_local_retention_bytes, log_local_retention_msKafka kind option remote_log_storage_system_enableOpenSearch kind option auth_failure_listenersOpenSearch kind Index State Management optionsKafkaKafka version 3.5Kafka spec property scheduled_rebalance_max_delay_msKafka spec property remote_log_storage_system_enableKafkaConnect spec property scheduled_rebalance_max_delay_msOpenSearch spec property openidKAFKA_SCHEMA_REGISTRY_HOST and KAFKA_SCHEMA_REGISTRY_PORT for KafkaKAFKA_CONNECT_HOST, KAFKA_CONNECT_PORT, KAFKA_REST_HOST and KAFKA_REST_PORT for Kafka. Thanks to @Dariuschunclean_leader_election_enable from KafkaTopic kind configKAFKA_SASL_PORT for Kafka kind if SASL authentication method is enabledredis options to datadog ServiceIntegrationCassandra version 3Kafka versions 3.1 and 3.4kafka_rest_config.producer_max_request_size optionkafka_mirrormaker.producer_compression_type optionclusterRole.create option to Helm chart. Thanks to @ryaneorthOpenSearch.spec.userConfig.idp_pemtrustedcas_content option. Specifies the PEM-encoded root certificate authority (CA) content for the SAML identity provider (IdP) server verification.ServiceIntegration kind SourceProjectName and DestinationProjectName fieldsServiceIntegration fields MaxLength validationServiceIntegration validation: multiple user configs cannot be setServiceIntegration, should not require destinationServiceName or sourceEndpointID fieldServiceIntegration, add missing external_aws_cloudwatch_metrics type config serializationServiceIntegration integration type listannotations and labels fields to connInfoSecretTargetOpenSearch.spec.userConfig.opensearch.search_max_buckets maximum to 65536plan as a required fieldminumim, maximum validations for number typeip_filter backward compatibilityclickhouseKafka.tables.data_format-property enum RawBLOB valueuserConfig.opensearch.email_sender_username validation patternlog_cleaner_min_cleanable_ratio minimum and maximum validation rules3.2, reached EOL10, reached EOLProjectVPC by ID to avoid conflictsProjectVPC deletion by exiting on DELETING statusClickhouseUser controllerClickhouseUser.spec.project and ClickhouseUser.spec.serviceName as immutablesignalfxAuthSecretRef fields marked as requireddatadog, kafka_connect, kafka_logs, metricsclickhouse_postgresql, clickhouse_kafka, clickhouse_kafka, logs, external_aws_cloudwatch_metricsKafkaTopic.Spec.topicName field. Unlike the metadata.name, supports additional characters and has a longer length. KafkaTopic.Spec.topicName replaces metadata.name in future releases and will be marked as required.false value for termination_protection propertymin_cleanable_dirty_ratio. Thanks to @TV2rdImportant: This release brings breaking changes to the userConfig property. After new charts are installed, update your existing instances manually using the kubectl edit command according to the API reference.
Note: It is now recommended to disable webhooks for Kubernetes version 1.25 and higher, as native CRD validation rules are used.
ip_filter field is now of object typeserviceIntegrations on service types. Only the read_replica type is available.min_cleanable_dirty_ratio config field supportspec.disk_space propertylinux/amd64 build. Thanks to @christoffer-eidenever from choices of maintenance dowdevelopment flag to configure logger's behaviormake generate-user-configs)genericServiceHandler to generalize service managementKafkaACL deletionProjectVPCRef property to Kafka, OpenSearch, Clickhouse and Redis kinds to get ProjectVPC ID when resource is readyProjectVPC deletion, deletes by ID first if possible, then tries by nameclient.Object storage update data lossfeatures:
improvements:
fixes:
improvements:
fixes:
features:
Use the following checks to help you troubleshoot the Aiven Operator for Kubernetes.
"},{"location":"troubleshooting.html#check-the-pods","title":"Check the Pods","text":"Verify that all the operator Pods are READY, and the STATUS is Running.
kubectl get pod -n aiven-operator-system\nThe output is similar to the following:
NAME READY STATUS RESTARTS AGE\naiven-operator-controller-manager-576d944499-ggttj 1/1 Running 0 12m\nVerify that the cert-manager Pods are also running.
kubectl get pod -n cert-manager\nThe output has the status:
NAME READY STATUS RESTARTS AGE\ncert-manager-7dd5854bb4-85cpv 1/1 Running 0 76s\ncert-manager-cainjector-64c949654c-n2z8l 1/1 Running 0 77s\ncert-manager-webhook-6bdffc7c9d-47w6z 1/1 Running 0 76s\nUse the following command to visualize all the logs from the operator.
kubectl logs -n aiven-operator-system -l control-plane=controller-manager\nkubectl get pod -n aiven-operator-system -l control-plane=controller-manager -o jsonpath=\"{.items[0].spec.containers[0].image}\"\nWe're always working to resolve problems that pop up in Aiven products. If your problem is listed below, we know about it and are working to fix it. If your problem isn't listed below, report it as an issue.
"},{"location":"troubleshooting.html#cert-manager","title":"cert-manager","text":""},{"location":"troubleshooting.html#issue","title":"Issue","text":"The following event appears on the operator Pod:
MountVolume.SetUp failed for volume \"cert\" : secret \"webhook-server-cert\" not found\nYou cannot run the operator.
"},{"location":"troubleshooting.html#solution","title":"Solution","text":"Make sure that cert-manager is up and running.
kubectl get pod -n cert-manager\nThe output shows the status of each cert-manager:
NAME READY STATUS RESTARTS AGE\ncert-manager-7dd5854bb4-85cpv 1/1 Running 0 76s\ncert-manager-cainjector-64c949654c-n2z8l 1/1 Running 0 77s\ncert-manager-webhook-6bdffc7c9d-47w6z 1/1 Running 0 76s\nAutogenerated from CRD files.
"},{"location":"api-reference/cassandra.html","title":"Cassandra","text":""},{"location":"api-reference/cassandra.html#usage-example","title":"Usage example","text":"ExampleapiVersion: aiven.io/v1alpha1\nkind: Cassandra\nmetadata:\n name: my-cassandra\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n connInfoSecretTarget:\n name: cassandra-secret\n prefix: MY_SECRET_PREFIX_\n annotations:\n foo: bar\n labels:\n baz: egg\n\n project: aiven-project-name\n cloudName: google-europe-west1\n plan: startup-4\n\n maintenanceWindowDow: sunday\n maintenanceWindowTime: 11:00:00\n\n userConfig:\n migrate_sstableloader: true\n public_access:\n prometheus: true\n ip_filter:\n - network: 0.0.0.0\n description: whatever\n - network: 10.20.0.0/16\nInfo
To create this resource, a Secret containing Aiven token must be created first.
Apply the resource with:
kubectl apply -f example.yaml\nVerify the newly created Cassandra:
kubectl get cassandras my-cassandra\nThe output is similar to the following:
Name Project Region Plan State \nmy-cassandra aiven-project-name google-europe-west1 startup-4 RUNNING \nTo view the details of the Secret, use the following command:
kubectl describe secret cassandra-secret\nYou can use the jq to quickly decode the Secret:
kubectl get secret cassandra-secret -o json | jq '.data | map_values(@base64d)'\nThe output is similar to the following:
{\n \"CASSANDRA_HOST\": \"<secret>\",\n \"CASSANDRA_PORT\": \"<secret>\",\n \"CASSANDRA_USER\": \"<secret>\",\n \"CASSANDRA_PASSWORD\": \"<secret>\",\n \"CASSANDRA_URI\": \"<secret>\",\n \"CASSANDRA_HOSTS\": \"<secret>\",\n \"CASSANDRA_CA_CERT\": \"<secret>\",\n}\nCassandra is the Schema for the cassandras API.
Exposes secret keys
CASSANDRA_HOST, CASSANDRA_PORT, CASSANDRA_USER, CASSANDRA_PASSWORD, CASSANDRA_URI, CASSANDRA_HOSTS, CASSANDRA_CA_CERT.
Required
apiVersion (string). Value aiven.io/v1alpha1.kind (string). Value Cassandra.metadata (object). Data that identifies the object, including a name string and optional namespace.spec (object). CassandraSpec defines the desired state of Cassandra. See below for nested schema.Appears on Cassandra.
CassandraSpec defines the desired state of Cassandra.
Required
plan (string, MaxLength: 128). Subscription plan.project (string, Immutable, Pattern: ^[a-zA-Z0-9_-]+$, MaxLength: 63). Identifies the project this resource belongs to.Optional
authSecretRef (object). Authentication reference to Aiven token in a secret. See below for nested schema.cloudName (string, MaxLength: 256). Cloud the service runs in.connInfoSecretTarget (object). Secret configuration. See below for nested schema.connInfoSecretTargetDisabled (boolean, Immutable). When true, the secret containing connection information will not be created, defaults to false. This field cannot be changed after resource creation.disk_space (string, Pattern: (?i)^[1-9][0-9]*(GiB|G)?$). The disk space of the service, possible values depend on the service type, the cloud provider and the project. Reducing will result in the service re-balancing. The removal of this field does not change the value.maintenanceWindowDow (string, Enum: monday, tuesday, wednesday, thursday, friday, saturday, sunday). Day of week when maintenance operations should be performed. One monday, tuesday, wednesday, etc.maintenanceWindowTime (string, MaxLength: 8). Time of day when maintenance operations should be performed. UTC time in HH:mm:ss format.projectVPCRef (object). ProjectVPCRef reference to ProjectVPC resource to use its ID as ProjectVPCID automatically. See below for nested schema.projectVpcId (string, MaxLength: 36). Identifier of the VPC the service should be in, if any.serviceIntegrations (array of objects, Immutable, MaxItems: 1). Service integrations to specify when creating a service. Not applied after initial service creation. See below for nested schema.tags (object, AdditionalProperties: string). Tags are key-value pairs that allow you to categorize services.technicalEmails (array of objects, MaxItems: 10). Defines the email addresses that will receive alerts about upcoming maintenance updates or warnings about service instability. See below for nested schema.terminationProtection (boolean). Prevent service from being deleted. It is recommended to have this enabled for all services.userConfig (object). Cassandra specific user configuration options. See below for nested schema.Appears on spec.
Authentication reference to Aiven token in a secret.
Required
key (string, MinLength: 1).name (string, MinLength: 1).Appears on spec.
Secret configuration.
Required
name (string, Immutable). Name of the secret resource to be created. By default, it is equal to the resource name.Optional
annotations (object, AdditionalProperties: string). Annotations added to the secret.labels (object, AdditionalProperties: string). Labels added to the secret.prefix (string). Prefix for the secret's keys. Added \"as is\" without any transformations. By default, is equal to the kind name in uppercase + underscore, e.g. KAFKA_, REDIS_, etc.Appears on spec.
ProjectVPCRef reference to ProjectVPC resource to use its ID as ProjectVPCID automatically.
Required
name (string, MinLength: 1).Optional
namespace (string, MinLength: 1).Appears on spec.
Service integrations to specify when creating a service. Not applied after initial service creation.
Required
integrationType (string, Enum: read_replica).sourceServiceName (string, MinLength: 1, MaxLength: 64).Appears on spec.
Defines the email addresses that will receive alerts about upcoming maintenance updates or warnings about service instability.
Required
email (string). Email address.Appears on spec.
Cassandra specific user configuration options.
Optional
additional_backup_regions (array of strings, MaxItems: 1). Deprecated. Additional Cloud Regions for Backup Replication.backup_hour (integer, Minimum: 0, Maximum: 23). The hour of day (in UTC) when backup for the service is started. New backup is only started if previous backup has already completed.backup_minute (integer, Minimum: 0, Maximum: 59). The minute of an hour when backup for the service is started. New backup is only started if previous backup has already completed.cassandra (object). cassandra configuration values. See below for nested schema.cassandra_version (string, Enum: 4, 4.1, Pattern: ^[0-9]+(\\.[0-9]+)?$). Cassandra version.ip_filter (array of objects, MaxItems: 1024). Allow incoming connections from CIDR address block, e.g. 10.20.0.0/16. See below for nested schema.migrate_sstableloader (boolean). Sets the service into migration mode enabling the sstableloader utility to be used to upload Cassandra data files. Available only on service create.private_access (object). Allow access to selected service ports from private networks. See below for nested schema.project_to_fork_from (string, Immutable, Pattern: ^[a-z][-a-z0-9]{0,63}$|^$, MaxLength: 63). Name of another project to fork a service from. This has effect only when a new service is being created.public_access (object). Allow access to selected service ports from the public Internet. See below for nested schema.service_log (boolean). Store logs for the service so that they are available in the HTTP API and console.service_to_fork_from (string, Immutable, Pattern: ^[a-z][-a-z0-9]{0,63}$|^$, MaxLength: 64). Name of another service to fork from. This has effect only when a new service is being created.service_to_join_with (string, Pattern: ^[a-z][-a-z0-9]{0,63}$, MaxLength: 64). When bootstrapping, instead of creating a new Cassandra cluster try to join an existing one from another service. Can only be set on service creation.static_ips (boolean). Use static public IP addresses.Appears on spec.userConfig.
cassandra configuration values.
Optional
batch_size_fail_threshold_in_kb (integer, Minimum: 1, Maximum: 1000000). Fail any multiple-partition batch exceeding this value. 50kb (10x warn threshold) by default.batch_size_warn_threshold_in_kb (integer, Minimum: 1, Maximum: 1000000). Log a warning message on any multiple-partition batch size exceeding this value.5kb per batch by default.Caution should be taken on increasing the size of this thresholdas it can lead to node instability.datacenter (string, MaxLength: 128). Name of the datacenter to which nodes of this service belong. Can be set only when creating the service.Appears on spec.userConfig.
Allow incoming connections from CIDR address block, e.g. 10.20.0.0/16.
Required
network (string, MaxLength: 43). CIDR address block.Optional
description (string, MaxLength: 1024). Description for IP filter list entry.Appears on spec.userConfig.
Allow access to selected service ports from private networks.
Required
prometheus (boolean). Allow clients to connect to prometheus with a DNS name that always resolves to the service's private IP addresses. Only available in certain network locations.Appears on spec.userConfig.
Allow access to selected service ports from the public Internet.
Required
prometheus (boolean). Allow clients to connect to prometheus from the public internet for service nodes that are in a project VPC or another type of private network.apiVersion: aiven.io/v1alpha1\nkind: Clickhouse\nmetadata:\n name: my-clickhouse\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n connInfoSecretTarget:\n name: my-clickhouse\n annotations:\n foo: bar\n labels:\n baz: egg\n\n tags:\n env: test\n instance: foo\n\n userConfig:\n ip_filter:\n - network: 0.0.0.0/32\n description: bar\n - network: 10.20.0.0/16\n\n project: my-aiven-project\n cloudName: google-europe-west1\n plan: startup-16\n\n maintenanceWindowDow: friday\n maintenanceWindowTime: 23:00:00\nInfo
To create this resource, a Secret containing Aiven token must be created first.
Apply the resource with:
kubectl apply -f example.yaml\nVerify the newly created Clickhouse:
kubectl get clickhouses my-clickhouse\nThe output is similar to the following:
Name Project Region Plan State \nmy-clickhouse my-aiven-project google-europe-west1 startup-16 RUNNING \nTo view the details of the Secret, use the following command:
kubectl describe secret my-clickhouse\nYou can use the jq to quickly decode the Secret:
kubectl get secret my-clickhouse -o json | jq '.data | map_values(@base64d)'\nThe output is similar to the following:
{\n \"CLICKHOUSE_HOST\": \"<secret>\",\n \"CLICKHOUSE_PORT\": \"<secret>\",\n \"CLICKHOUSE_USER\": \"<secret>\",\n \"CLICKHOUSE_PASSWORD\": \"<secret>\",\n}\nClickhouse is the Schema for the clickhouses API.
Exposes secret keys
CLICKHOUSE_HOST, CLICKHOUSE_PORT, CLICKHOUSE_USER, CLICKHOUSE_PASSWORD.
Required
apiVersion (string). Value aiven.io/v1alpha1.kind (string). Value Clickhouse.metadata (object). Data that identifies the object, including a name string and optional namespace.spec (object). ClickhouseSpec defines the desired state of Clickhouse. See below for nested schema.Appears on Clickhouse.
ClickhouseSpec defines the desired state of Clickhouse.
Required
plan (string, MaxLength: 128). Subscription plan.project (string, Immutable, Pattern: ^[a-zA-Z0-9_-]+$, MaxLength: 63). Identifies the project this resource belongs to.Optional
authSecretRef (object). Authentication reference to Aiven token in a secret. See below for nested schema.cloudName (string, MaxLength: 256). Cloud the service runs in.connInfoSecretTarget (object). Secret configuration. See below for nested schema.connInfoSecretTargetDisabled (boolean, Immutable). When true, the secret containing connection information will not be created, defaults to false. This field cannot be changed after resource creation.disk_space (string, Pattern: (?i)^[1-9][0-9]*(GiB|G)?$). The disk space of the service, possible values depend on the service type, the cloud provider and the project. Reducing will result in the service re-balancing. The removal of this field does not change the value.maintenanceWindowDow (string, Enum: monday, tuesday, wednesday, thursday, friday, saturday, sunday). Day of week when maintenance operations should be performed. One monday, tuesday, wednesday, etc.maintenanceWindowTime (string, MaxLength: 8). Time of day when maintenance operations should be performed. UTC time in HH:mm:ss format.projectVPCRef (object). ProjectVPCRef reference to ProjectVPC resource to use its ID as ProjectVPCID automatically. See below for nested schema.projectVpcId (string, MaxLength: 36). Identifier of the VPC the service should be in, if any.serviceIntegrations (array of objects, Immutable, MaxItems: 1). Service integrations to specify when creating a service. Not applied after initial service creation. See below for nested schema.tags (object, AdditionalProperties: string). Tags are key-value pairs that allow you to categorize services.technicalEmails (array of objects, MaxItems: 10). Defines the email addresses that will receive alerts about upcoming maintenance updates or warnings about service instability. See below for nested schema.terminationProtection (boolean). Prevent service from being deleted. It is recommended to have this enabled for all services.userConfig (object). OpenSearch specific user configuration options. See below for nested schema.Appears on spec.
Authentication reference to Aiven token in a secret.
Required
key (string, MinLength: 1).name (string, MinLength: 1).Appears on spec.
Secret configuration.
Required
name (string, Immutable). Name of the secret resource to be created. By default, it is equal to the resource name.Optional
annotations (object, AdditionalProperties: string). Annotations added to the secret.labels (object, AdditionalProperties: string). Labels added to the secret.prefix (string). Prefix for the secret's keys. Added \"as is\" without any transformations. By default, is equal to the kind name in uppercase + underscore, e.g. KAFKA_, REDIS_, etc.Appears on spec.
ProjectVPCRef reference to ProjectVPC resource to use its ID as ProjectVPCID automatically.
Required
name (string, MinLength: 1).Optional
namespace (string, MinLength: 1).Appears on spec.
Service integrations to specify when creating a service. Not applied after initial service creation.
Required
integrationType (string, Enum: read_replica).sourceServiceName (string, MinLength: 1, MaxLength: 64).Appears on spec.
Defines the email addresses that will receive alerts about upcoming maintenance updates or warnings about service instability.
Required
email (string). Email address.Appears on spec.
OpenSearch specific user configuration options.
Optional
additional_backup_regions (array of strings, MaxItems: 1). Deprecated. Additional Cloud Regions for Backup Replication.ip_filter (array of objects, MaxItems: 1024). Allow incoming connections from CIDR address block, e.g. 10.20.0.0/16. See below for nested schema.private_access (object). Allow access to selected service ports from private networks. See below for nested schema.privatelink_access (object). Allow access to selected service components through Privatelink. See below for nested schema.project_to_fork_from (string, Immutable, Pattern: ^[a-z][-a-z0-9]{0,63}$|^$, MaxLength: 63). Name of another project to fork a service from. This has effect only when a new service is being created.public_access (object). Allow access to selected service ports from the public Internet. See below for nested schema.service_log (boolean). Store logs for the service so that they are available in the HTTP API and console.service_to_fork_from (string, Immutable, Pattern: ^[a-z][-a-z0-9]{0,63}$|^$, MaxLength: 64). Name of another service to fork from. This has effect only when a new service is being created.static_ips (boolean). Use static public IP addresses.Appears on spec.userConfig.
Allow incoming connections from CIDR address block, e.g. 10.20.0.0/16.
Required
network (string, MaxLength: 43). CIDR address block.Optional
description (string, MaxLength: 1024). Description for IP filter list entry.Appears on spec.userConfig.
Allow access to selected service ports from private networks.
Optional
clickhouse (boolean). Allow clients to connect to clickhouse with a DNS name that always resolves to the service's private IP addresses. Only available in certain network locations.clickhouse_https (boolean). Allow clients to connect to clickhouse_https with a DNS name that always resolves to the service's private IP addresses. Only available in certain network locations.clickhouse_mysql (boolean). Allow clients to connect to clickhouse_mysql with a DNS name that always resolves to the service's private IP addresses. Only available in certain network locations.prometheus (boolean). Allow clients to connect to prometheus with a DNS name that always resolves to the service's private IP addresses. Only available in certain network locations.Appears on spec.userConfig.
Allow access to selected service components through Privatelink.
Optional
clickhouse (boolean). Enable clickhouse.clickhouse_https (boolean). Enable clickhouse_https.clickhouse_mysql (boolean). Enable clickhouse_mysql.prometheus (boolean). Enable prometheus.Appears on spec.userConfig.
Allow access to selected service ports from the public Internet.
Optional
clickhouse (boolean). Allow clients to connect to clickhouse from the public internet for service nodes that are in a project VPC or another type of private network.clickhouse_https (boolean). Allow clients to connect to clickhouse_https from the public internet for service nodes that are in a project VPC or another type of private network.clickhouse_mysql (boolean). Allow clients to connect to clickhouse_mysql from the public internet for service nodes that are in a project VPC or another type of private network.prometheus (boolean). Allow clients to connect to prometheus from the public internet for service nodes that are in a project VPC or another type of private network.apiVersion: aiven.io/v1alpha1\nkind: ClickhouseDatabase\nmetadata:\n name: my-db\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n project: my-aiven-project\n serviceName: my-clickhouse\n databaseName: example-db\nInfo
To create this resource, a Secret containing Aiven token must be created first.
Apply the resource with:
kubectl apply -f example.yaml\nVerify the newly created ClickhouseDatabase:
kubectl get clickhousedatabases my-db\nThe output is similar to the following:
Name Database name Service Name Project \nmy-db example-db my-clickhouse my-aiven-project \nClickhouseDatabase is the Schema for the databases API.
Required
apiVersion (string). Value aiven.io/v1alpha1.kind (string). Value ClickhouseDatabase.metadata (object). Data that identifies the object, including a name string and optional namespace.spec (object). ClickhouseDatabaseSpec defines the desired state of ClickhouseDatabase. See below for nested schema.Appears on ClickhouseDatabase.
ClickhouseDatabaseSpec defines the desired state of ClickhouseDatabase.
Required
project (string, Immutable, Pattern: ^[a-zA-Z0-9_-]+$, MaxLength: 63). Identifies the project this resource belongs to.serviceName (string, Immutable, Pattern: ^[a-z][-a-z0-9]+$, MaxLength: 63). Specifies the name of the service that this resource belongs to.Optional
authSecretRef (object). Authentication reference to Aiven token in a secret. See below for nested schema.databaseName (string, Immutable, MaxLength: 63). Specifies the Clickhouse database name. Defaults to metadata.name if omitted. Note: metadata.name is ASCII-only. For UTF-8 names, use spec.databaseName, but ASCII is advised for compatibility.Appears on spec.
Authentication reference to Aiven token in a secret.
Required
key (string, MinLength: 1).name (string, MinLength: 1).apiVersion: aiven.io/v1alpha1\nkind: ClickhouseGrant\nmetadata:\n name: demo-ch-grant\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n project: my-aiven-project\n serviceName: my-clickhouse\n\n privilegeGrants:\n - grantees:\n - user: user1\n - user: my-clickhouse-user-\ud83e\udd84\n privileges:\n - SELECT\n - INSERT\n database: my-db\n # If table is omitted, the privileges are granted on all tables in the database\n # If columns is omitted, the privileges are granted on all columns in the table\n - grantees:\n - role: my-role\n privileges:\n - SELECT\n database: my-db\n table: my-table\n columns:\n - col1\n - col2\n\n roleGrants:\n - roles:\n - other-role\n grantees:\n - user: my-user\n - role: my-role\napiVersion: aiven.io/v1alpha1\nkind: ClickhouseGrant\nmetadata:\n name: my-clickhouse-grant\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n project: aiven-project-name\n serviceName: my-clickhouse-service\n\n privilegeGrants:\n - grantees:\n - role: my-clickhouse-role\n privileges:\n - INSERT\n - SELECT\n - CREATE TABLE\n - CREATE VIEW\n database: my-clickhouse-db\n roleGrants:\n - grantees:\n - user: my-clickhouse-user\n roles:\n - my-clickhouse-role\n\n---\n\napiVersion: aiven.io/v1alpha1\nkind: Clickhouse\nmetadata:\n name: my-clickhouse-service\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n project: aiven-project-name\n cloudName: google-europe-west1\n plan: startup-16\n\n---\n\napiVersion: aiven.io/v1alpha1\nkind: ClickhouseDatabase\nmetadata:\n name: my-clickhouse-db\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n project: aiven-project-name\n serviceName: my-clickhouse-service\n\n---\n\napiVersion: aiven.io/v1alpha1\nkind: ClickhouseUser\nmetadata:\n name: my-clickhouse-user\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n project: aiven-project-name\n serviceName: my-clickhouse-service\n\n---\n\napiVersion: aiven.io/v1alpha1\nkind: ClickhouseRole\nmetadata:\n name: my-clickhouse-role\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n project: aiven-project-name\n serviceName: my-clickhouse-service\n role: my-clickhouse-role\nInfo
To create this resource, a Secret containing Aiven token must be created first.
Apply the resource with:
kubectl apply -f example.yaml\nVerify the newly created ClickhouseGrant:
kubectl get clickhousegrants demo-ch-grant\nThe output is similar to the following:
Name Project Service Name \ndemo-ch-grant my-aiven-project my-clickhouse \nClickhouseGrant is the Schema for the ClickhouseGrants API
Ambiguity in the GRANT syntax
Due to an ambiguity in the GRANT syntax in Clickhouse, you should not have users and roles with the same name. It is not clear if a grant refers to the user or the role.
Required
apiVersion (string). Value aiven.io/v1alpha1.kind (string). Value ClickhouseGrant.metadata (object). Data that identifies the object, including a name string and optional namespace.spec (object). ClickhouseGrantSpec defines the desired state of ClickhouseGrant. See below for nested schema.Appears on ClickhouseGrant.
ClickhouseGrantSpec defines the desired state of ClickhouseGrant.
Required
project (string, Immutable, Pattern: ^[a-zA-Z0-9_-]+$, MaxLength: 63). Identifies the project this resource belongs to.serviceName (string, Immutable, Pattern: ^[a-z][-a-z0-9]+$, MaxLength: 63). Specifies the name of the service that this resource belongs to.Optional
authSecretRef (object). Authentication reference to Aiven token in a secret. See below for nested schema.privilegeGrants (array of objects). Configuration to grant a privilege. Privileges not in the manifest are revoked. Existing privileges are retained; new ones are granted. See below for nested schema.roleGrants (array of objects). Configuration to grant a role. Role grants not in the manifest are revoked. Existing role grants are retained; new ones are granted. See below for nested schema.Appears on spec.
Authentication reference to Aiven token in a secret.
Required
key (string, MinLength: 1).name (string, MinLength: 1).Appears on spec.
Configuration to grant a privilege. Privileges not in the manifest are revoked. Existing privileges are retained; new ones are granted.
Required
database (string). The database that the grant refers to.grantees (array of objects, MinItems: 1). List of grantees (users or roles) to grant the privilege to. See below for nested schema.privileges (array of strings). The privileges to grant, i.e. INSERT, SELECT. See https://clickhouse.com/docs/en/sql-reference/statements/grant#assigning-role-syntax.Optional
columns (array of strings). The column that the grant refers to.table (string). The tables that the grant refers to. To grant a privilege on all tables in a database, omit this field instead of writing table: \"*\".withGrantOption (boolean). If true, then the grantee (user or role) get the permission to execute the GRANT query. Users can grant privileges of the same scope they have and less. See https://clickhouse.com/docs/en/sql-reference/statements/grant#granting-privilege-syntax.Appears on spec.privilegeGrants.
List of grantees (users or roles) to grant the privilege to.
Optional
role (string).user (string).Appears on spec.
Configuration to grant a role. Role grants not in the manifest are revoked. Existing role grants are retained; new ones are granted.
Required
grantees (array of objects, MinItems: 1). List of grantees (users or roles) to grant the privilege to. See below for nested schema.roles (array of strings, MinItems: 1). List of roles to grant to the grantees.Optional
withAdminOption (boolean). If true, the grant is executed with ADMIN OPTION privilege. See https://clickhouse.com/docs/en/sql-reference/statements/grant#admin-option.Appears on spec.roleGrants.
List of grantees (users or roles) to grant the privilege to.
Optional
role (string).user (string).apiVersion: aiven.io/v1alpha1\nkind: ClickhouseRole\nmetadata:\n name: my-role\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n project: my-aiven-project\n serviceName: my-clickhouse\n role: my-role\nInfo
To create this resource, a Secret containing Aiven token must be created first.
Apply the resource with:
kubectl apply -f example.yaml\nVerify the newly created ClickhouseRole:
kubectl get clickhouseroles my-role\nThe output is similar to the following:
Name Project Service Name Role \nmy-role my-aiven-project my-clickhouse my-role \nClickhouseRole is the Schema for the clickhouseroles API.
Required
apiVersion (string). Value aiven.io/v1alpha1.kind (string). Value ClickhouseRole.metadata (object). Data that identifies the object, including a name string and optional namespace.spec (object). ClickhouseRoleSpec defines the desired state of ClickhouseRole. See below for nested schema.Appears on ClickhouseRole.
ClickhouseRoleSpec defines the desired state of ClickhouseRole.
Required
project (string, Immutable, Pattern: ^[a-zA-Z0-9_-]+$, MaxLength: 63). Identifies the project this resource belongs to.role (string, Immutable, MaxLength: 255). The role that is to be created.serviceName (string, Immutable, Pattern: ^[a-z][-a-z0-9]+$, MaxLength: 63). Specifies the name of the service that this resource belongs to.Optional
authSecretRef (object). Authentication reference to Aiven token in a secret. See below for nested schema.Appears on spec.
Authentication reference to Aiven token in a secret.
Required
key (string, MinLength: 1).name (string, MinLength: 1).apiVersion: aiven.io/v1alpha1\nkind: ClickhouseUser\nmetadata:\n name: my-clickhouse-user\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n connInfoSecretTarget:\n name: clickhouse-user-secret\n annotations:\n foo: bar\n labels:\n baz: egg\n\n project: my-aiven-project\n serviceName: my-clickhouse\n username: example-username\n\n---\n\napiVersion: aiven.io/v1alpha1\nkind: Clickhouse\nmetadata:\n name: my-clickhouse\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n project: my-aiven-project\n cloudName: google-europe-west1\n plan: startup-16\nInfo
To create this resource, a Secret containing Aiven token must be created first.
Apply the resource with:
kubectl apply -f example.yaml\nVerify the newly created ClickhouseUser:
kubectl get clickhouseusers my-clickhouse-user\nThe output is similar to the following:
Name Username Service Name Project \nmy-clickhouse-user example-username my-clickhouse my-aiven-project \nTo view the details of the Secret, use the following command:
kubectl describe secret clickhouse-user-secret\nYou can use the jq to quickly decode the Secret:
kubectl get secret clickhouse-user-secret -o json | jq '.data | map_values(@base64d)'\nThe output is similar to the following:
{\n \"CLICKHOUSEUSER_HOST\": \"<secret>\",\n \"CLICKHOUSEUSER_PORT\": \"<secret>\",\n \"CLICKHOUSEUSER_USER\": \"<secret>\",\n \"CLICKHOUSEUSER_PASSWORD\": \"<secret>\",\n}\nClickhouseUser is the Schema for the clickhouseusers API.
Exposes secret keys
CLICKHOUSEUSER_HOST, CLICKHOUSEUSER_PORT, CLICKHOUSEUSER_USER, CLICKHOUSEUSER_PASSWORD.
Required
apiVersion (string). Value aiven.io/v1alpha1.kind (string). Value ClickhouseUser.metadata (object). Data that identifies the object, including a name string and optional namespace.spec (object). ClickhouseUserSpec defines the desired state of ClickhouseUser. See below for nested schema.Appears on ClickhouseUser.
ClickhouseUserSpec defines the desired state of ClickhouseUser.
Required
project (string, Immutable, Pattern: ^[a-zA-Z0-9_-]+$, MaxLength: 63). Identifies the project this resource belongs to.serviceName (string, Immutable, Pattern: ^[a-z][-a-z0-9]+$, MaxLength: 63). Specifies the name of the service that this resource belongs to.Optional
authSecretRef (object). Authentication reference to Aiven token in a secret. See below for nested schema.connInfoSecretTarget (object). Secret configuration. See below for nested schema.connInfoSecretTargetDisabled (boolean, Immutable). When true, the secret containing connection information will not be created, defaults to false. This field cannot be changed after resource creation.username (string, Immutable, MaxLength: 63). Name of the Clickhouse user. Defaults to metadata.name if omitted. Note: metadata.name is ASCII-only. For UTF-8 names, use spec.username, but ASCII is advised for compatibility.Appears on spec.
Authentication reference to Aiven token in a secret.
Required
key (string, MinLength: 1).name (string, MinLength: 1).Appears on spec.
Secret configuration.
Required
name (string, Immutable). Name of the secret resource to be created. By default, it is equal to the resource name.Optional
annotations (object, AdditionalProperties: string). Annotations added to the secret.labels (object, AdditionalProperties: string). Labels added to the secret.prefix (string). Prefix for the secret's keys. Added \"as is\" without any transformations. By default, is equal to the kind name in uppercase + underscore, e.g. KAFKA_, REDIS_, etc.apiVersion: aiven.io/v1alpha1\nkind: ConnectionPool\nmetadata:\n name: my-connection-pool\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n project: aiven-project-name\n serviceName: my-pg\n databaseName: my-database\n username: my-service-user\n poolMode: transaction\n poolSize: 25\n\n---\n\napiVersion: aiven.io/v1alpha1\nkind: PostgreSQL\nmetadata:\n name: my-pg\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n project: aiven-project-name\n cloudName: google-europe-west1\n plan: startup-4\n\n---\n\napiVersion: aiven.io/v1alpha1\nkind: Database\nmetadata:\n name: my-database\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n project: aiven-project-name\n serviceName: my-pg\n\n---\n\napiVersion: aiven.io/v1alpha1\nkind: ServiceUser\nmetadata:\n name: my-service-user\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n project: aiven-project-name\n serviceName: my-pg\nInfo
To create this resource, a Secret containing Aiven token must be created first.
Apply the resource with:
kubectl apply -f example.yaml\nVerify the newly created ConnectionPool:
kubectl get connectionpools my-connection-pool\nThe output is similar to the following:
Name Service Name Project Database Username Pool Size Pool Mode \nmy-connection-pool my-pg aiven-project-name my-database my-service-user 25 transaction \nConnectionPool is the Schema for the connectionpools API.
Exposes secret keys
CONNECTIONPOOL_NAME, CONNECTIONPOOL_HOST, CONNECTIONPOOL_PORT, CONNECTIONPOOL_DATABASE, CONNECTIONPOOL_USER, CONNECTIONPOOL_PASSWORD, CONNECTIONPOOL_SSLMODE, CONNECTIONPOOL_DATABASE_URI, CONNECTIONPOOL_CA_CERT.
Required
apiVersion (string). Value aiven.io/v1alpha1.kind (string). Value ConnectionPool.metadata (object). Data that identifies the object, including a name string and optional namespace.spec (object). ConnectionPoolSpec defines the desired state of ConnectionPool. See below for nested schema.Appears on ConnectionPool.
ConnectionPoolSpec defines the desired state of ConnectionPool.
Required
databaseName (string, MaxLength: 40). Name of the database the pool connects to.project (string, Immutable, Pattern: ^[a-zA-Z0-9_-]+$, MaxLength: 63). Identifies the project this resource belongs to.serviceName (string, Immutable, Pattern: ^[a-z][-a-z0-9]+$, MaxLength: 63). Specifies the name of the service that this resource belongs to.username (string, MaxLength: 64). Name of the service user used to connect to the database.Optional
authSecretRef (object). Authentication reference to Aiven token in a secret. See below for nested schema.connInfoSecretTarget (object). Secret configuration. See below for nested schema.connInfoSecretTargetDisabled (boolean, Immutable). When true, the secret containing connection information will not be created, defaults to false. This field cannot be changed after resource creation.poolMode (string, Enum: session, transaction, statement). Mode the pool operates in (session, transaction, statement).poolSize (integer). Number of connections the pool may create towards the backend server.Appears on spec.
Authentication reference to Aiven token in a secret.
Required
key (string, MinLength: 1).name (string, MinLength: 1).Appears on spec.
Secret configuration.
Required
name (string, Immutable). Name of the secret resource to be created. By default, it is equal to the resource name.Optional
annotations (object, AdditionalProperties: string). Annotations added to the secret.labels (object, AdditionalProperties: string). Labels added to the secret.prefix (string). Prefix for the secret's keys. Added \"as is\" without any transformations. By default, is equal to the kind name in uppercase + underscore, e.g. KAFKA_, REDIS_, etc.apiVersion: aiven.io/v1alpha1\nkind: Database\nmetadata:\n name: my-db\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n project: aiven-project-name\n serviceName: my-service\n\n lcCtype: en_US.UTF-8\n lcCollate: en_US.UTF-8\nInfo
To create this resource, a Secret containing Aiven token must be created first.
Apply the resource with:
kubectl apply -f example.yaml\nVerify the newly created Database:
kubectl get databases my-db\nThe output is similar to the following:
Name Project Service Name \nmy-db aiven-project-name my-service \nDatabase is the Schema for the databases API.
Required
apiVersion (string). Value aiven.io/v1alpha1.kind (string). Value Database.metadata (object). Data that identifies the object, including a name string and optional namespace.spec (object). DatabaseSpec defines the desired state of Database. See below for nested schema.Appears on Database.
DatabaseSpec defines the desired state of Database.
Required
project (string, Immutable, Pattern: ^[a-zA-Z0-9_-]+$, MaxLength: 63). Identifies the project this resource belongs to.serviceName (string, Immutable, Pattern: ^[a-z][-a-z0-9]+$, MaxLength: 63). Specifies the name of the service that this resource belongs to.Optional
authSecretRef (object). Authentication reference to Aiven token in a secret. See below for nested schema.lcCollate (string, Immutable, MaxLength: 128). Default string sort order (LC_COLLATE) of the database. Default value: en_US.UTF-8.lcCtype (string, Immutable, MaxLength: 128). Default character classification (LC_CTYPE) of the database. Default value: en_US.UTF-8.terminationProtection (boolean). It is a Kubernetes side deletion protections, which prevents the database from being deleted by Kubernetes. It is recommended to enable this for any production databases containing critical data.Appears on spec.
Authentication reference to Aiven token in a secret.
Required
key (string, MinLength: 1).name (string, MinLength: 1).apiVersion: aiven.io/v1alpha1\nkind: Grafana\nmetadata:\n name: my-grafana\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n connInfoSecretTarget:\n name: grafana-secret\n prefix: MY_SECRET_PREFIX_\n annotations:\n foo: bar\n labels:\n baz: egg\n\n project: my-aiven-project\n cloudName: google-europe-west1\n plan: startup-1\n\n maintenanceWindowDow: sunday\n maintenanceWindowTime: 11:00:00\n\n userConfig:\n public_access:\n grafana: true\n ip_filter:\n - network: 0.0.0.0\n description: whatever\n - network: 10.20.0.0/16\nInfo
To create this resource, a Secret containing Aiven token must be created first.
Apply the resource with:
kubectl apply -f example.yaml\nVerify the newly created Grafana:
kubectl get grafanas my-grafana\nThe output is similar to the following:
Name Project Region Plan State \nmy-grafana my-aiven-project google-europe-west1 startup-1 RUNNING \nTo view the details of the Secret, use the following command:
kubectl describe secret grafana-secret\nYou can use the jq to quickly decode the Secret:
kubectl get secret grafana-secret -o json | jq '.data | map_values(@base64d)'\nThe output is similar to the following:
{\n \"GRAFANA_HOST\": \"<secret>\",\n \"GRAFANA_PORT\": \"<secret>\",\n \"GRAFANA_USER\": \"<secret>\",\n \"GRAFANA_PASSWORD\": \"<secret>\",\n \"GRAFANA_URI\": \"<secret>\",\n \"GRAFANA_HOSTS\": \"<secret>\",\n}\nGrafana is the Schema for the grafanas API.
Exposes secret keys
GRAFANA_HOST, GRAFANA_PORT, GRAFANA_USER, GRAFANA_PASSWORD, GRAFANA_URI, GRAFANA_HOSTS.
Required
apiVersion (string). Value aiven.io/v1alpha1.kind (string). Value Grafana.metadata (object). Data that identifies the object, including a name string and optional namespace.spec (object). GrafanaSpec defines the desired state of Grafana. See below for nested schema.Appears on Grafana.
GrafanaSpec defines the desired state of Grafana.
Required
plan (string, MaxLength: 128). Subscription plan.project (string, Immutable, Pattern: ^[a-zA-Z0-9_-]+$, MaxLength: 63). Identifies the project this resource belongs to.Optional
authSecretRef (object). Authentication reference to Aiven token in a secret. See below for nested schema.cloudName (string, MaxLength: 256). Cloud the service runs in.connInfoSecretTarget (object). Secret configuration. See below for nested schema.connInfoSecretTargetDisabled (boolean, Immutable). When true, the secret containing connection information will not be created, defaults to false. This field cannot be changed after resource creation.disk_space (string, Pattern: (?i)^[1-9][0-9]*(GiB|G)?$). The disk space of the service, possible values depend on the service type, the cloud provider and the project. Reducing will result in the service re-balancing. The removal of this field does not change the value.maintenanceWindowDow (string, Enum: monday, tuesday, wednesday, thursday, friday, saturday, sunday). Day of week when maintenance operations should be performed. One monday, tuesday, wednesday, etc.maintenanceWindowTime (string, MaxLength: 8). Time of day when maintenance operations should be performed. UTC time in HH:mm:ss format.projectVPCRef (object). ProjectVPCRef reference to ProjectVPC resource to use its ID as ProjectVPCID automatically. See below for nested schema.projectVpcId (string, MaxLength: 36). Identifier of the VPC the service should be in, if any.serviceIntegrations (array of objects, Immutable, MaxItems: 1). Service integrations to specify when creating a service. Not applied after initial service creation. See below for nested schema.tags (object, AdditionalProperties: string). Tags are key-value pairs that allow you to categorize services.technicalEmails (array of objects, MaxItems: 10). Defines the email addresses that will receive alerts about upcoming maintenance updates or warnings about service instability. See below for nested schema.terminationProtection (boolean). Prevent service from being deleted. It is recommended to have this enabled for all services.userConfig (object). Cassandra specific user configuration options. See below for nested schema.Appears on spec.
Authentication reference to Aiven token in a secret.
Required
key (string, MinLength: 1).name (string, MinLength: 1).Appears on spec.
Secret configuration.
Required
name (string, Immutable). Name of the secret resource to be created. By default, it is equal to the resource name.Optional
annotations (object, AdditionalProperties: string). Annotations added to the secret.labels (object, AdditionalProperties: string). Labels added to the secret.prefix (string). Prefix for the secret's keys. Added \"as is\" without any transformations. By default, is equal to the kind name in uppercase + underscore, e.g. KAFKA_, REDIS_, etc.Appears on spec.
ProjectVPCRef reference to ProjectVPC resource to use its ID as ProjectVPCID automatically.
Required
name (string, MinLength: 1).Optional
namespace (string, MinLength: 1).Appears on spec.
Service integrations to specify when creating a service. Not applied after initial service creation.
Required
integrationType (string, Enum: read_replica).sourceServiceName (string, MinLength: 1, MaxLength: 64).Appears on spec.
Defines the email addresses that will receive alerts about upcoming maintenance updates or warnings about service instability.
Required
email (string). Email address.Appears on spec.
Cassandra specific user configuration options.
Optional
additional_backup_regions (array of strings, MaxItems: 1). Additional Cloud Regions for Backup Replication.alerting_enabled (boolean). Enable or disable Grafana legacy alerting functionality. This should not be enabled with unified_alerting_enabled.alerting_error_or_timeout (string, Enum: alerting, keep_state). Default error or timeout setting for new alerting rules.alerting_max_annotations_to_keep (integer, Minimum: 0, Maximum: 1000000). Max number of alert annotations that Grafana stores. 0 (default) keeps all alert annotations.alerting_nodata_or_nullvalues (string, Enum: alerting, no_data, keep_state, ok). Default value for 'no data or null values' for new alerting rules.allow_embedding (boolean). Allow embedding Grafana dashboards with iframe/frame/object/embed tags. Disabled by default to limit impact of clickjacking.auth_azuread (object). Azure AD OAuth integration. See below for nested schema.auth_basic_enabled (boolean). Enable or disable basic authentication form, used by Grafana built-in login.auth_generic_oauth (object). Generic OAuth integration. See below for nested schema.auth_github (object). Github Auth integration. See below for nested schema.auth_gitlab (object). GitLab Auth integration. See below for nested schema.auth_google (object). Google Auth integration. See below for nested schema.cookie_samesite (string, Enum: lax, strict, none). Cookie SameSite attribute: strict prevents sending cookie for cross-site requests, effectively disabling direct linking from other sites to Grafana. lax is the default value.custom_domain (string, MaxLength: 255). Serve the web frontend using a custom CNAME pointing to the Aiven DNS name.dashboard_previews_enabled (boolean). This feature is new in Grafana 9 and is quite resource intensive. It may cause low-end plans to work more slowly while the dashboard previews are rendering.dashboards_min_refresh_interval (string, Pattern: ^[0-9]+(ms|s|m|h|d)$, MaxLength: 16). Signed sequence of decimal numbers, followed by a unit suffix (ms, s, m, h, d), e.g. 30s, 1h.dashboards_versions_to_keep (integer, Minimum: 1, Maximum: 100). Dashboard versions to keep per dashboard.dataproxy_send_user_header (boolean). Send X-Grafana-User header to data source.dataproxy_timeout (integer, Minimum: 15, Maximum: 90). Timeout for data proxy requests in seconds.date_formats (object). Grafana date format specifications. See below for nested schema.disable_gravatar (boolean). Set to true to disable gravatar. Defaults to false (gravatar is enabled).editors_can_admin (boolean). Editors can manage folders, teams and dashboards created by them.external_image_storage (object). External image store settings. See below for nested schema.google_analytics_ua_id (string, Pattern: ^(G|UA|YT|MO)-[a-zA-Z0-9-]+$, MaxLength: 64). Google Analytics ID.ip_filter (array of objects, MaxItems: 1024). Allow incoming connections from CIDR address block, e.g. 10.20.0.0/16. See below for nested schema.metrics_enabled (boolean). Enable Grafana /metrics endpoint.oauth_allow_insecure_email_lookup (boolean). Enforce user lookup based on email instead of the unique ID provided by the IdP.private_access (object). Allow access to selected service ports from private networks. See below for nested schema.privatelink_access (object). Allow access to selected service components through Privatelink. See below for nested schema.project_to_fork_from (string, Immutable, Pattern: ^[a-z][-a-z0-9]{0,63}$|^$, MaxLength: 63). Name of another project to fork a service from. This has effect only when a new service is being created.public_access (object). Allow access to selected service ports from the public Internet. See below for nested schema.recovery_basebackup_name (string, Pattern: ^[a-zA-Z0-9-_:.]+$, MaxLength: 128). Name of the basebackup to restore in forked service.service_log (boolean). Store logs for the service so that they are available in the HTTP API and console.service_to_fork_from (string, Immutable, Pattern: ^[a-z][-a-z0-9]{0,63}$|^$, MaxLength: 64). Name of another service to fork from. This has effect only when a new service is being created.smtp_server (object). SMTP server settings. See below for nested schema.static_ips (boolean). Use static public IP addresses.unified_alerting_enabled (boolean). Enable or disable Grafana unified alerting functionality. By default this is enabled and any legacy alerts will be migrated on upgrade to Grafana 9+. To stay on legacy alerting, set unified_alerting_enabled to false and alerting_enabled to true. See https://grafana.com/docs/grafana/latest/alerting/set-up/migrating-alerts/ for more details.user_auto_assign_org (boolean). Auto-assign new users on signup to main organization. Defaults to false.user_auto_assign_org_role (string, Enum: Viewer, Admin, Editor). Set role for new signups. Defaults to Viewer.viewers_can_edit (boolean). Users with view-only permission can edit but not save dashboards.Appears on spec.userConfig.
Azure AD OAuth integration.
Required
auth_url (string, MaxLength: 2048). Authorization URL.client_id (string, Pattern: ^[\\040-\\176]+$, MaxLength: 1024). Client ID from provider.client_secret (string, Pattern: ^[\\040-\\176]+$, MaxLength: 1024). Client secret from provider.token_url (string, MaxLength: 2048). Token URL.Optional
allow_sign_up (boolean). Automatically sign-up users on successful sign-in.allowed_domains (array of strings, MaxItems: 50). Allowed domains.allowed_groups (array of strings, MaxItems: 50). Require users to belong to one of given groups.Appears on spec.userConfig.
Generic OAuth integration.
Required
api_url (string, MaxLength: 2048). API URL.auth_url (string, MaxLength: 2048). Authorization URL.client_id (string, Pattern: ^[\\040-\\176]+$, MaxLength: 1024). Client ID from provider.client_secret (string, Pattern: ^[\\040-\\176]+$, MaxLength: 1024). Client secret from provider.token_url (string, MaxLength: 2048). Token URL.Optional
allow_sign_up (boolean). Automatically sign-up users on successful sign-in.allowed_domains (array of strings, MaxItems: 50). Allowed domains.allowed_organizations (array of strings, MaxItems: 50). Require user to be member of one of the listed organizations.auto_login (boolean). Allow users to bypass the login screen and automatically log in.name (string, Pattern: ^[a-zA-Z0-9_\\- ]+$, MaxLength: 128). Name of the OAuth integration.scopes (array of strings, MaxItems: 50). OAuth scopes.Appears on spec.userConfig.
Github Auth integration.
Required
client_id (string, Pattern: ^[\\040-\\176]+$, MaxLength: 1024). Client ID from provider.client_secret (string, Pattern: ^[\\040-\\176]+$, MaxLength: 1024). Client secret from provider.Optional
allow_sign_up (boolean). Automatically sign-up users on successful sign-in.allowed_organizations (array of strings, MaxItems: 50). Require users to belong to one of given organizations.auto_login (boolean). Allow users to bypass the login screen and automatically log in.skip_org_role_sync (boolean). Stop automatically syncing user roles.team_ids (array of integers, MaxItems: 50). Require users to belong to one of given team IDs.Appears on spec.userConfig.
GitLab Auth integration.
Required
allowed_groups (array of strings, MaxItems: 50). Require users to belong to one of given groups.client_id (string, Pattern: ^[\\040-\\176]+$, MaxLength: 1024). Client ID from provider.client_secret (string, Pattern: ^[\\040-\\176]+$, MaxLength: 1024). Client secret from provider.Optional
allow_sign_up (boolean). Automatically sign-up users on successful sign-in.api_url (string, MaxLength: 2048). API URL. This only needs to be set when using self hosted GitLab.auth_url (string, MaxLength: 2048). Authorization URL. This only needs to be set when using self hosted GitLab.token_url (string, MaxLength: 2048). Token URL. This only needs to be set when using self hosted GitLab.Appears on spec.userConfig.
Google Auth integration.
Required
allowed_domains (array of strings, MaxItems: 64). Domains allowed to sign-in to this Grafana.client_id (string, Pattern: ^[\\040-\\176]+$, MaxLength: 1024). Client ID from provider.client_secret (string, Pattern: ^[\\040-\\176]+$, MaxLength: 1024). Client secret from provider.Optional
allow_sign_up (boolean). Automatically sign-up users on successful sign-in.Appears on spec.userConfig.
Grafana date format specifications.
Optional
default_timezone (string, MaxLength: 64). Default time zone for user preferences. Value browser uses browser local time zone.full_date (string, MaxLength: 128). Moment.js style format string for cases where full date is shown.interval_day (string, MaxLength: 128). Moment.js style format string used when a time requiring day accuracy is shown.interval_hour (string, MaxLength: 128). Moment.js style format string used when a time requiring hour accuracy is shown.interval_minute (string, MaxLength: 128). Moment.js style format string used when a time requiring minute accuracy is shown.interval_month (string, MaxLength: 128). Moment.js style format string used when a time requiring month accuracy is shown.interval_second (string, MaxLength: 128). Moment.js style format string used when a time requiring second accuracy is shown.interval_year (string, MaxLength: 128). Moment.js style format string used when a time requiring year accuracy is shown.Appears on spec.userConfig.
External image store settings.
Required
access_key (string, Pattern: ^[A-Z0-9]+$, MaxLength: 4096). S3 access key. Requires permissions to the S3 bucket for the s3:PutObject and s3:PutObjectAcl actions.bucket_url (string, MaxLength: 2048). Bucket URL for S3.provider (string, Enum: s3). Provider type.secret_key (string, Pattern: ^[A-Za-z0-9/+=]+$, MaxLength: 4096). S3 secret key.Appears on spec.userConfig.
Allow incoming connections from CIDR address block, e.g. 10.20.0.0/16.
Required
network (string, MaxLength: 43). CIDR address block.Optional
description (string, MaxLength: 1024). Description for IP filter list entry.Appears on spec.userConfig.
Allow access to selected service ports from private networks.
Required
grafana (boolean). Allow clients to connect to grafana with a DNS name that always resolves to the service's private IP addresses. Only available in certain network locations.Appears on spec.userConfig.
Allow access to selected service components through Privatelink.
Required
grafana (boolean). Enable grafana.Appears on spec.userConfig.
Allow access to selected service ports from the public Internet.
Required
grafana (boolean). Allow clients to connect to grafana from the public internet for service nodes that are in a project VPC or another type of private network.Appears on spec.userConfig.
SMTP server settings.
Required
from_address (string, MaxLength: 319). Address used for sending emails.host (string, MaxLength: 255). Server hostname or IP.port (integer, Minimum: 1, Maximum: 65535). SMTP server port.Optional
from_name (string, Pattern: ^[^\\x00-\\x1F]+$, MaxLength: 128). Name used in outgoing emails, defaults to Grafana.password (string, Pattern: ^[^\\x00-\\x1F]+$, MaxLength: 255). Password for SMTP authentication.skip_verify (boolean). Skip verifying server certificate. Defaults to false.starttls_policy (string, Enum: OpportunisticStartTLS, MandatoryStartTLS, NoStartTLS). Either OpportunisticStartTLS, MandatoryStartTLS or NoStartTLS. Default is OpportunisticStartTLS.username (string, Pattern: ^[^\\x00-\\x1F]+$, MaxLength: 255). Username for SMTP authentication.apiVersion: aiven.io/v1alpha1\nkind: Kafka\nmetadata:\n name: my-kafka\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n connInfoSecretTarget:\n name: kafka-secret\n prefix: MY_SECRET_PREFIX_\n annotations:\n foo: bar\n labels:\n baz: egg\n\n project: my-aiven-project\n cloudName: google-europe-west1\n plan: startup-2\n\n maintenanceWindowDow: friday\n maintenanceWindowTime: 23:00:00\nInfo
To create this resource, a Secret containing Aiven token must be created first.
Apply the resource with:
kubectl apply -f example.yaml\nVerify the newly created Kafka:
kubectl get kafkas my-kafka\nThe output is similar to the following:
Name Project Region Plan State \nmy-kafka my-aiven-project google-europe-west1 startup-2 RUNNING \nTo view the details of the Secret, use the following command:
kubectl describe secret kafka-secret\nYou can use the jq to quickly decode the Secret:
kubectl get secret kafka-secret -o json | jq '.data | map_values(@base64d)'\nThe output is similar to the following:
{\n \"KAFKA_HOST\": \"<secret>\",\n \"KAFKA_PORT\": \"<secret>\",\n \"KAFKA_USERNAME\": \"<secret>\",\n \"KAFKA_PASSWORD\": \"<secret>\",\n \"KAFKA_ACCESS_CERT\": \"<secret>\",\n \"KAFKA_ACCESS_KEY\": \"<secret>\",\n \"KAFKA_SASL_HOST\": \"<secret>\",\n \"KAFKA_SASL_PORT\": \"<secret>\",\n \"KAFKA_SCHEMA_REGISTRY_HOST\": \"<secret>\",\n \"KAFKA_SCHEMA_REGISTRY_PORT\": \"<secret>\",\n \"KAFKA_CONNECT_HOST\": \"<secret>\",\n \"KAFKA_CONNECT_PORT\": \"<secret>\",\n \"KAFKA_REST_HOST\": \"<secret>\",\n \"KAFKA_REST_PORT\": \"<secret>\",\n \"KAFKA_CA_CERT\": \"<secret>\",\n}\nKafka is the Schema for the kafkas API.
Exposes secret keys
KAFKA_HOST, KAFKA_PORT, KAFKA_USERNAME, KAFKA_PASSWORD, KAFKA_ACCESS_CERT, KAFKA_ACCESS_KEY, KAFKA_SASL_HOST, KAFKA_SASL_PORT, KAFKA_SCHEMA_REGISTRY_HOST, KAFKA_SCHEMA_REGISTRY_PORT, KAFKA_CONNECT_HOST, KAFKA_CONNECT_PORT, KAFKA_REST_HOST, KAFKA_REST_PORT, KAFKA_CA_CERT.
Required
apiVersion (string). Value aiven.io/v1alpha1.kind (string). Value Kafka.metadata (object). Data that identifies the object, including a name string and optional namespace.spec (object). KafkaSpec defines the desired state of Kafka. See below for nested schema.Appears on Kafka.
KafkaSpec defines the desired state of Kafka.
Required
plan (string, MaxLength: 128). Subscription plan.project (string, Immutable, Pattern: ^[a-zA-Z0-9_-]+$, MaxLength: 63). Identifies the project this resource belongs to.Optional
authSecretRef (object). Authentication reference to Aiven token in a secret. See below for nested schema.cloudName (string, MaxLength: 256). Cloud the service runs in.connInfoSecretTarget (object). Secret configuration. See below for nested schema.connInfoSecretTargetDisabled (boolean, Immutable). When true, the secret containing connection information will not be created, defaults to false. This field cannot be changed after resource creation.disk_space (string, Pattern: (?i)^[1-9][0-9]*(GiB|G)?$). The disk space of the service, possible values depend on the service type, the cloud provider and the project. Reducing will result in the service re-balancing. The removal of this field does not change the value.karapace (boolean). Switch the service to use Karapace for schema registry and REST proxy.maintenanceWindowDow (string, Enum: monday, tuesday, wednesday, thursday, friday, saturday, sunday). Day of week when maintenance operations should be performed. One monday, tuesday, wednesday, etc.maintenanceWindowTime (string, MaxLength: 8). Time of day when maintenance operations should be performed. UTC time in HH:mm:ss format.projectVPCRef (object). ProjectVPCRef reference to ProjectVPC resource to use its ID as ProjectVPCID automatically. See below for nested schema.projectVpcId (string, MaxLength: 36). Identifier of the VPC the service should be in, if any.serviceIntegrations (array of objects, Immutable, MaxItems: 1). Service integrations to specify when creating a service. Not applied after initial service creation. See below for nested schema.tags (object, AdditionalProperties: string). Tags are key-value pairs that allow you to categorize services.technicalEmails (array of objects, MaxItems: 10). Defines the email addresses that will receive alerts about upcoming maintenance updates or warnings about service instability. See below for nested schema.terminationProtection (boolean). Prevent service from being deleted. It is recommended to have this enabled for all services.userConfig (object). Kafka specific user configuration options. See below for nested schema.Appears on spec.
Authentication reference to Aiven token in a secret.
Required
key (string, MinLength: 1).name (string, MinLength: 1).Appears on spec.
Secret configuration.
Required
name (string, Immutable). Name of the secret resource to be created. By default, it is equal to the resource name.Optional
annotations (object, AdditionalProperties: string). Annotations added to the secret.labels (object, AdditionalProperties: string). Labels added to the secret.prefix (string). Prefix for the secret's keys. Added \"as is\" without any transformations. By default, is equal to the kind name in uppercase + underscore, e.g. KAFKA_, REDIS_, etc.Appears on spec.
ProjectVPCRef reference to ProjectVPC resource to use its ID as ProjectVPCID automatically.
Required
name (string, MinLength: 1).Optional
namespace (string, MinLength: 1).Appears on spec.
Service integrations to specify when creating a service. Not applied after initial service creation.
Required
integrationType (string, Enum: read_replica).sourceServiceName (string, MinLength: 1, MaxLength: 64).Appears on spec.
Defines the email addresses that will receive alerts about upcoming maintenance updates or warnings about service instability.
Required
email (string). Email address.Appears on spec.
Kafka specific user configuration options.
Optional
additional_backup_regions (array of strings, MaxItems: 1). Deprecated. Additional Cloud Regions for Backup Replication.aiven_kafka_topic_messages (boolean). Allow access to read Kafka topic messages in the Aiven Console and REST API.custom_domain (string, MaxLength: 255). Serve the web frontend using a custom CNAME pointing to the Aiven DNS name.ip_filter (array of objects, MaxItems: 1024). Allow incoming connections from CIDR address block, e.g. 10.20.0.0/16. See below for nested schema.kafka (object). Kafka broker configuration values. See below for nested schema.kafka_authentication_methods (object). Kafka authentication methods. See below for nested schema.kafka_connect (boolean). Enable Kafka Connect service.kafka_connect_config (object). Kafka Connect configuration values. See below for nested schema.kafka_connect_secret_providers. See below for } (array of objects). Configure external secret providers in order to reference external secrets in connector configuration. Currently Hashicorp Vault (provider: vault, auth_method: token) and AWS Secrets Manager (provider: aws, auth_method: credentials) are supported. Secrets can be referenced in connector config with ${::nested schema. kafka_rest (boolean). Enable Kafka-REST service.kafka_rest_authorization (boolean). Enable authorization in Kafka-REST service.kafka_rest_config (object). Kafka REST configuration. See below for nested schema.kafka_version (string, Enum: 3.4, 3.5, 3.6, 3.7). Kafka major version.letsencrypt_sasl_privatelink (boolean). Use Letsencrypt CA for Kafka SASL via Privatelink.private_access (object). Allow access to selected service ports from private networks. See below for nested schema.privatelink_access (object). Allow access to selected service components through Privatelink. See below for nested schema.public_access (object). Allow access to selected service ports from the public Internet. See below for nested schema.schema_registry (boolean). Enable Schema-Registry service.schema_registry_config (object). Schema Registry configuration. See below for nested schema.service_log (boolean). Store logs for the service so that they are available in the HTTP API and console.static_ips (boolean). Use static public IP addresses.tiered_storage (object). Tiered storage configuration. See below for nested schema.Appears on spec.userConfig.
Allow incoming connections from CIDR address block, e.g. 10.20.0.0/16.
Required
network (string, MaxLength: 43). CIDR address block.Optional
description (string, MaxLength: 1024). Description for IP filter list entry.Appears on spec.userConfig.
Kafka broker configuration values.
Optional
auto_create_topics_enable (boolean). Enable auto creation of topics.compression_type (string, Enum: gzip, snappy, lz4, zstd, uncompressed, producer). Specify the final compression type for a given topic. This configuration accepts the standard compression codecs (gzip, snappy, lz4, zstd). It additionally accepts uncompressed which is equivalent to no compression; and producer which means retain the original compression codec set by the producer.connections_max_idle_ms (integer, Minimum: 1000, Maximum: 3600000). Idle connections timeout: the server socket processor threads close the connections that idle for longer than this.default_replication_factor (integer, Minimum: 1, Maximum: 10). Replication factor for autocreated topics.group_initial_rebalance_delay_ms (integer, Minimum: 0, Maximum: 300000). The amount of time, in milliseconds, the group coordinator will wait for more consumers to join a new group before performing the first rebalance. A longer delay means potentially fewer rebalances, but increases the time until processing begins. The default value for this is 3 seconds. During development and testing it might be desirable to set this to 0 in order to not delay test execution time.group_max_session_timeout_ms (integer, Minimum: 0, Maximum: 1800000). The maximum allowed session timeout for registered consumers. Longer timeouts give consumers more time to process messages in between heartbeats at the cost of a longer time to detect failures.group_min_session_timeout_ms (integer, Minimum: 0, Maximum: 60000). The minimum allowed session timeout for registered consumers. Longer timeouts give consumers more time to process messages in between heartbeats at the cost of a longer time to detect failures.log_cleaner_delete_retention_ms (integer, Minimum: 0, Maximum: 315569260000). How long are delete records retained?.log_cleaner_max_compaction_lag_ms (integer, Minimum: 30000). The maximum amount of time message will remain uncompacted. Only applicable for logs that are being compacted.log_cleaner_min_cleanable_ratio (number, Minimum: 0.2, Maximum: 0.9). Controls log compactor frequency. Larger value means more frequent compactions but also more space wasted for logs. Consider setting log.cleaner.max.compaction.lag.ms to enforce compactions sooner, instead of setting a very high value for this option.log_cleaner_min_compaction_lag_ms (integer, Minimum: 0). The minimum time a message will remain uncompacted in the log. Only applicable for logs that are being compacted.log_cleanup_policy (string, Enum: delete, compact, compact,delete). The default cleanup policy for segments beyond the retention window.log_flush_interval_messages (integer, Minimum: 1). The number of messages accumulated on a log partition before messages are flushed to disk.log_flush_interval_ms (integer, Minimum: 0). The maximum time in ms that a message in any topic is kept in memory before flushed to disk. If not set, the value in log.flush.scheduler.interval.ms is used.log_index_interval_bytes (integer, Minimum: 0, Maximum: 104857600). The interval with which Kafka adds an entry to the offset index.log_index_size_max_bytes (integer, Minimum: 1048576, Maximum: 104857600). The maximum size in bytes of the offset index.log_local_retention_bytes (integer, Minimum: -2). The maximum size of local log segments that can grow for a partition before it gets eligible for deletion. If set to -2, the value of log.retention.bytes is used. The effective value should always be less than or equal to log.retention.bytes value.log_local_retention_ms (integer, Minimum: -2). The number of milliseconds to keep the local log segments before it gets eligible for deletion. If set to -2, the value of log.retention.ms is used. The effective value should always be less than or equal to log.retention.ms value.log_message_downconversion_enable (boolean). This configuration controls whether down-conversion of message formats is enabled to satisfy consume requests.log_message_timestamp_difference_max_ms (integer, Minimum: 0). The maximum difference allowed between the timestamp when a broker receives a message and the timestamp specified in the message.log_message_timestamp_type (string, Enum: CreateTime, LogAppendTime). Define whether the timestamp in the message is message create time or log append time.log_preallocate (boolean). Should pre allocate file when create new segment?.log_retention_bytes (integer, Minimum: -1). The maximum size of the log before deleting messages.log_retention_hours (integer, Minimum: -1, Maximum: 2147483647). The number of hours to keep a log file before deleting it.log_retention_ms (integer, Minimum: -1). The number of milliseconds to keep a log file before deleting it (in milliseconds), If not set, the value in log.retention.minutes is used. If set to -1, no time limit is applied.log_roll_jitter_ms (integer, Minimum: 0). The maximum jitter to subtract from logRollTimeMillis (in milliseconds). If not set, the value in log.roll.jitter.hours is used.log_roll_ms (integer, Minimum: 1). The maximum time before a new log segment is rolled out (in milliseconds).log_segment_bytes (integer, Minimum: 10485760, Maximum: 1073741824). The maximum size of a single log file.log_segment_delete_delay_ms (integer, Minimum: 0, Maximum: 3600000). The amount of time to wait before deleting a file from the filesystem.max_connections_per_ip (integer, Minimum: 256, Maximum: 2147483647). The maximum number of connections allowed from each ip address (defaults to 2147483647).max_incremental_fetch_session_cache_slots (integer, Minimum: 1000, Maximum: 10000). The maximum number of incremental fetch sessions that the broker will maintain.message_max_bytes (integer, Minimum: 0, Maximum: 100001200). The maximum size of message that the server can receive.min_insync_replicas (integer, Minimum: 1, Maximum: 7). When a producer sets acks to all (or -1), min.insync.replicas specifies the minimum number of replicas that must acknowledge a write for the write to be considered successful.num_partitions (integer, Minimum: 1, Maximum: 1000). Number of partitions for autocreated topics.offsets_retention_minutes (integer, Minimum: 1, Maximum: 2147483647). Log retention window in minutes for offsets topic.producer_purgatory_purge_interval_requests (integer, Minimum: 10, Maximum: 10000). The purge interval (in number of requests) of the producer request purgatory(defaults to 1000).replica_fetch_max_bytes (integer, Minimum: 1048576, Maximum: 104857600). The number of bytes of messages to attempt to fetch for each partition (defaults to 1048576). This is not an absolute maximum, if the first record batch in the first non-empty partition of the fetch is larger than this value, the record batch will still be returned to ensure that progress can be made.replica_fetch_response_max_bytes (integer, Minimum: 10485760, Maximum: 1048576000). Maximum bytes expected for the entire fetch response (defaults to 10485760). Records are fetched in batches, and if the first record batch in the first non-empty partition of the fetch is larger than this value, the record batch will still be returned to ensure that progress can be made. As such, this is not an absolute maximum.sasl_oauthbearer_expected_audience (string, Pattern: ^[^\\r\\n]*$, MaxLength: 128). The (optional) comma-delimited setting for the broker to use to verify that the JWT was issued for one of the expected audiences.sasl_oauthbearer_expected_issuer (string, Pattern: ^[^\\r\\n]*$, MaxLength: 128). Optional setting for the broker to use to verify that the JWT was created by the expected issuer.sasl_oauthbearer_jwks_endpoint_url (string, MaxLength: 2048). OIDC JWKS endpoint URL. By setting this the SASL SSL OAuth2/OIDC authentication is enabled. See also other options for SASL OAuth2/OIDC.sasl_oauthbearer_sub_claim_name (string, Pattern: ^[^\\r\\n]*$, MaxLength: 128). Name of the scope from which to extract the subject claim from the JWT. Defaults to sub.socket_request_max_bytes (integer, Minimum: 10485760, Maximum: 209715200). The maximum number of bytes in a socket request (defaults to 104857600).transaction_partition_verification_enable (boolean). Enable verification that checks that the partition has been added to the transaction before writing transactional records to the partition.transaction_remove_expired_transaction_cleanup_interval_ms (integer, Minimum: 600000, Maximum: 3600000). The interval at which to remove transactions that have expired due to transactional.id.expiration.ms passing (defaults to 3600000 (1 hour)).transaction_state_log_segment_bytes (integer, Minimum: 1048576, Maximum: 2147483647). The transaction topic segment bytes should be kept relatively small in order to facilitate faster log compaction and cache loads (defaults to 104857600 (100 mebibytes)).Appears on spec.userConfig.
Kafka authentication methods.
Optional
certificate (boolean). Enable certificate/SSL authentication.sasl (boolean). Enable SASL authentication.Appears on spec.userConfig.
Kafka Connect configuration values.
Optional
connector_client_config_override_policy (string, Enum: None, All). Defines what client configurations can be overridden by the connector. Default is None.consumer_auto_offset_reset (string, Enum: earliest, latest). What to do when there is no initial offset in Kafka or if the current offset does not exist any more on the server. Default is earliest.consumer_fetch_max_bytes (integer, Minimum: 1048576, Maximum: 104857600). Records are fetched in batches by the consumer, and if the first record batch in the first non-empty partition of the fetch is larger than this value, the record batch will still be returned to ensure that the consumer can make progress. As such, this is not a absolute maximum.consumer_isolation_level (string, Enum: read_uncommitted, read_committed). Transaction read isolation level. read_uncommitted is the default, but read_committed can be used if consume-exactly-once behavior is desired.consumer_max_partition_fetch_bytes (integer, Minimum: 1048576, Maximum: 104857600). Records are fetched in batches by the consumer.If the first record batch in the first non-empty partition of the fetch is larger than this limit, the batch will still be returned to ensure that the consumer can make progress.consumer_max_poll_interval_ms (integer, Minimum: 1, Maximum: 2147483647). The maximum delay in milliseconds between invocations of poll() when using consumer group management (defaults to 300000).consumer_max_poll_records (integer, Minimum: 1, Maximum: 10000). The maximum number of records returned in a single call to poll() (defaults to 500).offset_flush_interval_ms (integer, Minimum: 1, Maximum: 100000000). The interval at which to try committing offsets for tasks (defaults to 60000).offset_flush_timeout_ms (integer, Minimum: 1, Maximum: 2147483647). Maximum number of milliseconds to wait for records to flush and partition offset data to be committed to offset storage before cancelling the process and restoring the offset data to be committed in a future attempt (defaults to 5000).producer_batch_size (integer, Minimum: 0, Maximum: 5242880). This setting gives the upper bound of the batch size to be sent. If there are fewer than this many bytes accumulated for this partition, the producer will linger for the linger.ms time waiting for more records to show up. A batch size of zero will disable batching entirely (defaults to 16384).producer_buffer_memory (integer, Minimum: 5242880, Maximum: 134217728). The total bytes of memory the producer can use to buffer records waiting to be sent to the broker (defaults to 33554432).producer_compression_type (string, Enum: gzip, snappy, lz4, zstd, none). Specify the default compression type for producers. This configuration accepts the standard compression codecs (gzip, snappy, lz4, zstd). It additionally accepts none which is the default and equivalent to no compression.producer_linger_ms (integer, Minimum: 0, Maximum: 5000). This setting gives the upper bound on the delay for batching: once there is batch.size worth of records for a partition it will be sent immediately regardless of this setting, however if there are fewer than this many bytes accumulated for this partition the producer will linger for the specified time waiting for more records to show up. Defaults to 0.producer_max_request_size (integer, Minimum: 131072, Maximum: 67108864). This setting will limit the number of record batches the producer will send in a single request to avoid sending huge requests.scheduled_rebalance_max_delay_ms (integer, Minimum: 0, Maximum: 600000). The maximum delay that is scheduled in order to wait for the return of one or more departed workers before rebalancing and reassigning their connectors and tasks to the group. During this period the connectors and tasks of the departed workers remain unassigned. Defaults to 5 minutes.session_timeout_ms (integer, Minimum: 1, Maximum: 2147483647). The timeout in milliseconds used to detect failures when using Kafka\u2019s group management facilities (defaults to 10000).Appears on spec.userConfig.
Configure external secret providers in order to reference external secrets in connector configuration. Currently Hashicorp Vault (provider: vault, auth_method: token) and AWS Secrets Manager (provider: aws, auth_method: credentials) are supported. Secrets can be referenced in connector config with ${::}.
Required
name (string). Name of the secret provider. Used to reference secrets in connector config.Optional
aws (object). AWS config for Secret Provider. See below for nested schema.vault (object). Vault Config for Secret Provider. See below for nested schema.Appears on spec.userConfig.kafka_connect_secret_providers.
AWS config for Secret Provider.
Required
auth_method (string, Enum: credentials). Auth method of the vault secret provider.region (string, MaxLength: 64). Region used to lookup secrets with AWS SecretManager.Optional
access_key (string, MaxLength: 128). Access key used to authenticate with aws.secret_key (string, MaxLength: 128). Secret key used to authenticate with aws.Appears on spec.userConfig.kafka_connect_secret_providers.
Vault Config for Secret Provider.
Required
address (string, MinLength: 1, MaxLength: 65536). Address of the Vault server.auth_method (string, Enum: token). Auth method of the vault secret provider.Optional
engine_version (integer, Enum: 1, 2). KV Secrets Engine version of the Vault server instance.token (string, MaxLength: 256). Token used to authenticate with vault and auth method token.Appears on spec.userConfig.
Kafka REST configuration.
Optional
consumer_enable_auto_commit (boolean). If true the consumer's offset will be periodically committed to Kafka in the background.consumer_request_max_bytes (integer, Minimum: 0, Maximum: 671088640). Maximum number of bytes in unencoded message keys and values by a single request.consumer_request_timeout_ms (integer, Enum: 1000, 15000, 30000, Minimum: 1000, Maximum: 30000). The maximum total time to wait for messages for a request if the maximum number of messages has not yet been reached.name_strategy (string, Enum: topic_name, record_name, topic_record_name). Name strategy to use when selecting subject for storing schemas.name_strategy_validation (boolean). If true, validate that given schema is registered under expected subject name by the used name strategy when producing messages.producer_acks (string, Enum: all, -1, 0, 1). The number of acknowledgments the producer requires the leader to have received before considering a request complete. If set to all or -1, the leader will wait for the full set of in-sync replicas to acknowledge the record.producer_compression_type (string, Enum: gzip, snappy, lz4, zstd, none). Specify the default compression type for producers. This configuration accepts the standard compression codecs (gzip, snappy, lz4, zstd). It additionally accepts none which is the default and equivalent to no compression.producer_linger_ms (integer, Minimum: 0, Maximum: 5000). Wait for up to the given delay to allow batching records together.producer_max_request_size (integer, Minimum: 0, Maximum: 2147483647). The maximum size of a request in bytes. Note that Kafka broker can also cap the record batch size.simpleconsumer_pool_size_max (integer, Minimum: 10, Maximum: 250). Maximum number of SimpleConsumers that can be instantiated per broker.Appears on spec.userConfig.
Allow access to selected service ports from private networks.
Optional
kafka (boolean). Allow clients to connect to kafka with a DNS name that always resolves to the service's private IP addresses. Only available in certain network locations.kafka_connect (boolean). Allow clients to connect to kafka_connect with a DNS name that always resolves to the service's private IP addresses. Only available in certain network locations.kafka_rest (boolean). Allow clients to connect to kafka_rest with a DNS name that always resolves to the service's private IP addresses. Only available in certain network locations.prometheus (boolean). Allow clients to connect to prometheus with a DNS name that always resolves to the service's private IP addresses. Only available in certain network locations.schema_registry (boolean). Allow clients to connect to schema_registry with a DNS name that always resolves to the service's private IP addresses. Only available in certain network locations.Appears on spec.userConfig.
Allow access to selected service components through Privatelink.
Optional
jolokia (boolean). Enable jolokia.kafka (boolean). Enable kafka.kafka_connect (boolean). Enable kafka_connect.kafka_rest (boolean). Enable kafka_rest.prometheus (boolean). Enable prometheus.schema_registry (boolean). Enable schema_registry.Appears on spec.userConfig.
Allow access to selected service ports from the public Internet.
Optional
kafka (boolean). Allow clients to connect to kafka from the public internet for service nodes that are in a project VPC or another type of private network.kafka_connect (boolean). Allow clients to connect to kafka_connect from the public internet for service nodes that are in a project VPC or another type of private network.kafka_rest (boolean). Allow clients to connect to kafka_rest from the public internet for service nodes that are in a project VPC or another type of private network.prometheus (boolean). Allow clients to connect to prometheus from the public internet for service nodes that are in a project VPC or another type of private network.schema_registry (boolean). Allow clients to connect to schema_registry from the public internet for service nodes that are in a project VPC or another type of private network.Appears on spec.userConfig.
Schema Registry configuration.
Optional
leader_eligibility (boolean). If true, Karapace / Schema Registry on the service nodes can participate in leader election. It might be needed to disable this when the schemas topic is replicated to a secondary cluster and Karapace / Schema Registry there must not participate in leader election. Defaults to true.topic_name (string, MinLength: 1, MaxLength: 249). The durable single partition topic that acts as the durable log for the data. This topic must be compacted to avoid losing data due to retention policy. Please note that changing this configuration in an existing Schema Registry / Karapace setup leads to previous schemas being inaccessible, data encoded with them potentially unreadable and schema ID sequence put out of order. It's only possible to do the switch while Schema Registry / Karapace is disabled. Defaults to _schemas.Appears on spec.userConfig.
Tiered storage configuration.
Optional
enabled (boolean). Whether to enable the tiered storage functionality.local_cache (object). Deprecated. Local cache configuration. See below for nested schema.Appears on spec.userConfig.tiered_storage.
Deprecated. Local cache configuration.
Required
size (integer, Minimum: 1, Maximum: 107374182400). Deprecated. Local cache size in bytes.apiVersion: aiven.io/v1alpha1\nkind: KafkaACL\nmetadata:\n name: my-kafka-acl\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n project: my-aiven-project\n serviceName: my-kafka\n topic: my-topic\n username: my-user\n permission: admin\nInfo
To create this resource, a Secret containing Aiven token must be created first.
Apply the resource with:
kubectl apply -f example.yaml\nVerify the newly created KafkaACL:
kubectl get kafkaacls my-kafka-acl\nThe output is similar to the following:
Name Service Name Project Username Permission Topic \nmy-kafka-acl my-kafka my-aiven-project my-user admin my-topic \nKafkaACL is the Schema for the kafkaacls API.
Required
apiVersion (string). Value aiven.io/v1alpha1.kind (string). Value KafkaACL.metadata (object). Data that identifies the object, including a name string and optional namespace.spec (object). KafkaACLSpec defines the desired state of KafkaACL. See below for nested schema.Appears on KafkaACL.
KafkaACLSpec defines the desired state of KafkaACL.
Required
permission (string, Enum: admin, read, readwrite, write). Kafka permission to grant (admin, read, readwrite, write).project (string, Immutable, Pattern: ^[a-zA-Z0-9_-]+$, MaxLength: 63). Identifies the project this resource belongs to.serviceName (string, Immutable, Pattern: ^[a-z][-a-z0-9]+$, MaxLength: 63). Specifies the name of the service that this resource belongs to.topic (string). Topic name pattern for the ACL entry.username (string). Username pattern for the ACL entry.Optional
authSecretRef (object). Authentication reference to Aiven token in a secret. See below for nested schema.Appears on spec.
Authentication reference to Aiven token in a secret.
Required
key (string, MinLength: 1).name (string, MinLength: 1).apiVersion: aiven.io/v1alpha1\nkind: KafkaConnect\nmetadata:\n name: my-kafka-connect\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n project: my-aiven-project\n cloudName: google-europe-west1\n plan: business-4\n\n userConfig:\n kafka_connect:\n consumer_isolation_level: read_committed\n public_access:\n kafka_connect: true\nInfo
To create this resource, a Secret containing Aiven token must be created first.
Apply the resource with:
kubectl apply -f example.yaml\nVerify the newly created KafkaConnect:
kubectl get kafkaconnects my-kafka-connect\nThe output is similar to the following:
Name Project Region Plan State \nmy-kafka-connect my-aiven-project google-europe-west1 business-4 RUNNING \nKafkaConnect is the Schema for the kafkaconnects API.
Required
apiVersion (string). Value aiven.io/v1alpha1.kind (string). Value KafkaConnect.metadata (object). Data that identifies the object, including a name string and optional namespace.spec (object). KafkaConnectSpec defines the desired state of KafkaConnect. See below for nested schema.Appears on KafkaConnect.
KafkaConnectSpec defines the desired state of KafkaConnect.
Required
plan (string, MaxLength: 128). Subscription plan.project (string, Immutable, Pattern: ^[a-zA-Z0-9_-]+$, MaxLength: 63). Identifies the project this resource belongs to.Optional
authSecretRef (object). Authentication reference to Aiven token in a secret. See below for nested schema.cloudName (string, MaxLength: 256). Cloud the service runs in.maintenanceWindowDow (string, Enum: monday, tuesday, wednesday, thursday, friday, saturday, sunday). Day of week when maintenance operations should be performed. One monday, tuesday, wednesday, etc.maintenanceWindowTime (string, MaxLength: 8). Time of day when maintenance operations should be performed. UTC time in HH:mm:ss format.projectVPCRef (object). ProjectVPCRef reference to ProjectVPC resource to use its ID as ProjectVPCID automatically. See below for nested schema.projectVpcId (string, MaxLength: 36). Identifier of the VPC the service should be in, if any.serviceIntegrations (array of objects, Immutable, MaxItems: 1). Service integrations to specify when creating a service. Not applied after initial service creation. See below for nested schema.tags (object, AdditionalProperties: string). Tags are key-value pairs that allow you to categorize services.technicalEmails (array of objects, MaxItems: 10). Defines the email addresses that will receive alerts about upcoming maintenance updates or warnings about service instability. See below for nested schema.terminationProtection (boolean). Prevent service from being deleted. It is recommended to have this enabled for all services.userConfig (object). KafkaConnect specific user configuration options. See below for nested schema.Appears on spec.
Authentication reference to Aiven token in a secret.
Required
key (string, MinLength: 1).name (string, MinLength: 1).Appears on spec.
ProjectVPCRef reference to ProjectVPC resource to use its ID as ProjectVPCID automatically.
Required
name (string, MinLength: 1).Optional
namespace (string, MinLength: 1).Appears on spec.
Service integrations to specify when creating a service. Not applied after initial service creation.
Required
integrationType (string, Enum: read_replica).sourceServiceName (string, MinLength: 1, MaxLength: 64).Appears on spec.
Defines the email addresses that will receive alerts about upcoming maintenance updates or warnings about service instability.
Required
email (string). Email address.Appears on spec.
KafkaConnect specific user configuration options.
Optional
additional_backup_regions (array of strings, MaxItems: 1). Deprecated. Additional Cloud Regions for Backup Replication.ip_filter (array of objects, MaxItems: 1024). Allow incoming connections from CIDR address block, e.g. 10.20.0.0/16. See below for nested schema.kafka_connect (object). Kafka Connect configuration values. See below for nested schema.private_access (object). Allow access to selected service ports from private networks. See below for nested schema.privatelink_access (object). Allow access to selected service components through Privatelink. See below for nested schema.public_access (object). Allow access to selected service ports from the public Internet. See below for nested schema.secret_providers. See below for } (array of objects). Configure external secret providers in order to reference external secrets in connector configuration. Currently Hashicorp Vault (provider: vault, auth_method: token) and AWS Secrets Manager (provider: aws, auth_method: credentials) are supported. Secrets can be referenced in connector config with ${::nested schema. service_log (boolean). Store logs for the service so that they are available in the HTTP API and console.static_ips (boolean). Use static public IP addresses.Appears on spec.userConfig.
Allow incoming connections from CIDR address block, e.g. 10.20.0.0/16.
Required
network (string, MaxLength: 43). CIDR address block.Optional
description (string, MaxLength: 1024). Description for IP filter list entry.Appears on spec.userConfig.
Kafka Connect configuration values.
Optional
connector_client_config_override_policy (string, Enum: None, All). Defines what client configurations can be overridden by the connector. Default is None.consumer_auto_offset_reset (string, Enum: earliest, latest). What to do when there is no initial offset in Kafka or if the current offset does not exist any more on the server. Default is earliest.consumer_fetch_max_bytes (integer, Minimum: 1048576, Maximum: 104857600). Records are fetched in batches by the consumer, and if the first record batch in the first non-empty partition of the fetch is larger than this value, the record batch will still be returned to ensure that the consumer can make progress. As such, this is not a absolute maximum.consumer_isolation_level (string, Enum: read_uncommitted, read_committed). Transaction read isolation level. read_uncommitted is the default, but read_committed can be used if consume-exactly-once behavior is desired.consumer_max_partition_fetch_bytes (integer, Minimum: 1048576, Maximum: 104857600). Records are fetched in batches by the consumer.If the first record batch in the first non-empty partition of the fetch is larger than this limit, the batch will still be returned to ensure that the consumer can make progress.consumer_max_poll_interval_ms (integer, Minimum: 1, Maximum: 2147483647). The maximum delay in milliseconds between invocations of poll() when using consumer group management (defaults to 300000).consumer_max_poll_records (integer, Minimum: 1, Maximum: 10000). The maximum number of records returned in a single call to poll() (defaults to 500).offset_flush_interval_ms (integer, Minimum: 1, Maximum: 100000000). The interval at which to try committing offsets for tasks (defaults to 60000).offset_flush_timeout_ms (integer, Minimum: 1, Maximum: 2147483647). Maximum number of milliseconds to wait for records to flush and partition offset data to be committed to offset storage before cancelling the process and restoring the offset data to be committed in a future attempt (defaults to 5000).producer_batch_size (integer, Minimum: 0, Maximum: 5242880). This setting gives the upper bound of the batch size to be sent. If there are fewer than this many bytes accumulated for this partition, the producer will linger for the linger.ms time waiting for more records to show up. A batch size of zero will disable batching entirely (defaults to 16384).producer_buffer_memory (integer, Minimum: 5242880, Maximum: 134217728). The total bytes of memory the producer can use to buffer records waiting to be sent to the broker (defaults to 33554432).producer_compression_type (string, Enum: gzip, snappy, lz4, zstd, none). Specify the default compression type for producers. This configuration accepts the standard compression codecs (gzip, snappy, lz4, zstd). It additionally accepts none which is the default and equivalent to no compression.producer_linger_ms (integer, Minimum: 0, Maximum: 5000). This setting gives the upper bound on the delay for batching: once there is batch.size worth of records for a partition it will be sent immediately regardless of this setting, however if there are fewer than this many bytes accumulated for this partition the producer will linger for the specified time waiting for more records to show up. Defaults to 0.producer_max_request_size (integer, Minimum: 131072, Maximum: 67108864). This setting will limit the number of record batches the producer will send in a single request to avoid sending huge requests.scheduled_rebalance_max_delay_ms (integer, Minimum: 0, Maximum: 600000). The maximum delay that is scheduled in order to wait for the return of one or more departed workers before rebalancing and reassigning their connectors and tasks to the group. During this period the connectors and tasks of the departed workers remain unassigned. Defaults to 5 minutes.session_timeout_ms (integer, Minimum: 1, Maximum: 2147483647). The timeout in milliseconds used to detect failures when using Kafka\u2019s group management facilities (defaults to 10000).Appears on spec.userConfig.
Allow access to selected service ports from private networks.
Optional
kafka_connect (boolean). Allow clients to connect to kafka_connect with a DNS name that always resolves to the service's private IP addresses. Only available in certain network locations.prometheus (boolean). Allow clients to connect to prometheus with a DNS name that always resolves to the service's private IP addresses. Only available in certain network locations.Appears on spec.userConfig.
Allow access to selected service components through Privatelink.
Optional
jolokia (boolean). Enable jolokia.kafka_connect (boolean). Enable kafka_connect.prometheus (boolean). Enable prometheus.Appears on spec.userConfig.
Allow access to selected service ports from the public Internet.
Optional
kafka_connect (boolean). Allow clients to connect to kafka_connect from the public internet for service nodes that are in a project VPC or another type of private network.prometheus (boolean). Allow clients to connect to prometheus from the public internet for service nodes that are in a project VPC or another type of private network.Appears on spec.userConfig.
Configure external secret providers in order to reference external secrets in connector configuration. Currently Hashicorp Vault (provider: vault, auth_method: token) and AWS Secrets Manager (provider: aws, auth_method: credentials) are supported. Secrets can be referenced in connector config with ${::}.
Required
name (string). Name of the secret provider. Used to reference secrets in connector config.Optional
aws (object). AWS config for Secret Provider. See below for nested schema.vault (object). Vault Config for Secret Provider. See below for nested schema.Appears on spec.userConfig.secret_providers.
AWS config for Secret Provider.
Required
auth_method (string, Enum: credentials). Auth method of the vault secret provider.region (string, MaxLength: 64). Region used to lookup secrets with AWS SecretManager.Optional
access_key (string, MaxLength: 128). Access key used to authenticate with aws.secret_key (string, MaxLength: 128). Secret key used to authenticate with aws.Appears on spec.userConfig.secret_providers.
Vault Config for Secret Provider.
Required
address (string, MinLength: 1, MaxLength: 65536). Address of the Vault server.auth_method (string, Enum: token). Auth method of the vault secret provider.Optional
engine_version (integer, Enum: 1, 2). KV Secrets Engine version of the Vault server instance.token (string, MaxLength: 256). Token used to authenticate with vault and auth method token.KafkaConnector is the Schema for the kafkaconnectors API.
Required
apiVersion (string). Value aiven.io/v1alpha1.kind (string). Value KafkaConnector.metadata (object). Data that identifies the object, including a name string and optional namespace.spec (object). KafkaConnectorSpec defines the desired state of KafkaConnector. See below for nested schema.Appears on KafkaConnector.
KafkaConnectorSpec defines the desired state of KafkaConnector.
Required
connectorClass (string, MaxLength: 1024). The Java class of the connector.project (string, Immutable, Pattern: ^[a-zA-Z0-9_-]+$, MaxLength: 63). Identifies the project this resource belongs to.serviceName (string, Immutable, Pattern: ^[a-z][-a-z0-9]+$, MaxLength: 63). Specifies the name of the service that this resource belongs to.userConfig (object, AdditionalProperties: string). The connector specific configuration To build config values from secret the template function {{ fromSecret \"name\" \"key\" }} is provided when interpreting the keys.Optional
authSecretRef (object). Authentication reference to Aiven token in a secret. See below for nested schema.Appears on spec.
Authentication reference to Aiven token in a secret.
Required
key (string, MinLength: 1).name (string, MinLength: 1).apiVersion: aiven.io/v1alpha1\nkind: KafkaSchema\nmetadata:\n name: my-schema\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n project: my-aiven-project\n serviceName: my-kafka\n subjectName: mny-subject\n compatibilityLevel: BACKWARD\n schema: |\n {\n \"doc\": \"example_doc\",\n \"fields\": [{\n \"default\": 5,\n \"doc\": \"field_doc\",\n \"name\": \"field_name\",\n \"namespace\": \"field_namespace\",\n \"type\": \"int\"\n }],\n \"name\": \"example_name\",\n \"namespace\": \"example_namespace\",\n \"type\": \"record\"\n }\nInfo
To create this resource, a Secret containing Aiven token must be created first.
Apply the resource with:
kubectl apply -f example.yaml\nVerify the newly created KafkaSchema:
kubectl get kafkaschemas my-schema\nThe output is similar to the following:
Name Service Name Project Subject Compatibility Level Version \nmy-schema my-kafka my-aiven-project mny-subject BACKWARD <version> \nKafkaSchema is the Schema for the kafkaschemas API.
Required
apiVersion (string). Value aiven.io/v1alpha1.kind (string). Value KafkaSchema.metadata (object). Data that identifies the object, including a name string and optional namespace.spec (object). KafkaSchemaSpec defines the desired state of KafkaSchema. See below for nested schema.Appears on KafkaSchema.
KafkaSchemaSpec defines the desired state of KafkaSchema.
Required
project (string, Immutable, Pattern: ^[a-zA-Z0-9_-]+$, MaxLength: 63). Identifies the project this resource belongs to.schema (string). Kafka Schema configuration should be a valid Avro Schema JSON format.serviceName (string, Immutable, Pattern: ^[a-z][-a-z0-9]+$, MaxLength: 63). Specifies the name of the service that this resource belongs to.subjectName (string, MaxLength: 63). Kafka Schema Subject name.Optional
authSecretRef (object). Authentication reference to Aiven token in a secret. See below for nested schema.compatibilityLevel (string, Enum: BACKWARD, BACKWARD_TRANSITIVE, FORWARD, FORWARD_TRANSITIVE, FULL, FULL_TRANSITIVE, NONE). Kafka Schemas compatibility level.Appears on spec.
Authentication reference to Aiven token in a secret.
Required
key (string, MinLength: 1).name (string, MinLength: 1).apiVersion: aiven.io/v1alpha1\nkind: KafkaSchemaRegistryACL\nmetadata:\n name: my-kafka-schema-registry-acl\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n project: aiven-project-name\n serviceName: my-kafka\n resource: Subject:my-topic\n username: my-user\n permission: schema_registry_read\nInfo
To create this resource, a Secret containing Aiven token must be created first.
Apply the resource with:
kubectl apply -f example.yaml\nVerify the newly created KafkaSchemaRegistryACL:
kubectl get kafkaschemaregistryacls my-kafka-schema-registry-acl\nThe output is similar to the following:
Name Project Service Name Resource Username State \nmy-kafka-schema-registry-acl aiven-project-name my-kafka Subject:my-topic my-user RUNNING \nKafkaSchemaRegistryACL is the Schema for the kafkaschemaregistryacls API.
Required
apiVersion (string). Value aiven.io/v1alpha1.kind (string). Value KafkaSchemaRegistryACL.metadata (object). Data that identifies the object, including a name string and optional namespace.spec (object). KafkaSchemaRegistryACLSpec defines the desired state of KafkaSchemaRegistryACL. See below for nested schema.Appears on KafkaSchemaRegistryACL.
KafkaSchemaRegistryACLSpec defines the desired state of KafkaSchemaRegistryACL.
Required
permission (string, Enum: schema_registry_read, schema_registry_write, Immutable).project (string, Immutable, Pattern: ^[a-zA-Z0-9_-]+$, MaxLength: 63). Identifies the project this resource belongs to.resource (string, Immutable, MaxLength: 249). Resource name pattern for the Schema Registry ACL entry.serviceName (string, Immutable, Pattern: ^[a-z][-a-z0-9]+$, MaxLength: 63). Specifies the name of the service that this resource belongs to.username (string, Immutable, MaxLength: 64). Username pattern for the ACL entry.Optional
authSecretRef (object). Authentication reference to Aiven token in a secret. See below for nested schema.Appears on spec.
Authentication reference to Aiven token in a secret.
Required
key (string, MinLength: 1).name (string, MinLength: 1).apiVersion: aiven.io/v1alpha1\nkind: KafkaTopic\nmetadata:\n name: kafka-topic\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n project: my-aiven-project\n serviceName: my-kafka\n topicName: my-kafka-topic\n\n replication: 2\n partitions: 1\n\n config:\n min_cleanable_dirty_ratio: 0.2\nInfo
To create this resource, a Secret containing Aiven token must be created first.
Apply the resource with:
kubectl apply -f example.yaml\nVerify the newly created KafkaTopic:
kubectl get kafkatopics kafka-topic\nThe output is similar to the following:
Name Service Name Project Partitions Replication \nkafka-topic my-kafka my-aiven-project 1 2 \nKafkaTopic is the Schema for the kafkatopics API.
Required
apiVersion (string). Value aiven.io/v1alpha1.kind (string). Value KafkaTopic.metadata (object). Data that identifies the object, including a name string and optional namespace.spec (object). KafkaTopicSpec defines the desired state of KafkaTopic. See below for nested schema.Appears on KafkaTopic.
KafkaTopicSpec defines the desired state of KafkaTopic.
Required
partitions (integer, Minimum: 1, Maximum: 1000000). Number of partitions to create in the topic.project (string, Immutable, Pattern: ^[a-zA-Z0-9_-]+$, MaxLength: 63). Identifies the project this resource belongs to.replication (integer, Minimum: 2). Replication factor for the topic.serviceName (string, Immutable, Pattern: ^[a-z][-a-z0-9]+$, MaxLength: 63). Specifies the name of the service that this resource belongs to.Optional
authSecretRef (object). Authentication reference to Aiven token in a secret. See below for nested schema.config (object). Kafka topic configuration. See below for nested schema.tags (array of objects). Kafka topic tags. See below for nested schema.termination_protection (boolean). It is a Kubernetes side deletion protections, which prevents the kafka topic from being deleted by Kubernetes. It is recommended to enable this for any production databases containing critical data.topicName (string, Immutable, MinLength: 1, MaxLength: 249). Topic name. If provided, is used instead of metadata.name. This field supports additional characters, has a longer length, and will replace metadata.name in future releases.Appears on spec.
Authentication reference to Aiven token in a secret.
Required
key (string, MinLength: 1).name (string, MinLength: 1).Appears on spec.
Kafka topic configuration.
Optional
cleanup_policy (string). cleanup.policy value.compression_type (string). compression.type value.delete_retention_ms (integer). delete.retention.ms value.file_delete_delay_ms (integer). file.delete.delay.ms value.flush_messages (integer). flush.messages value.flush_ms (integer). flush.ms value.index_interval_bytes (integer). index.interval.bytes value.local_retention_bytes (integer). local.retention.bytes value.local_retention_ms (integer). local.retention.ms value.max_compaction_lag_ms (integer). max.compaction.lag.ms value.max_message_bytes (integer). max.message.bytes value.message_downconversion_enable (boolean). message.downconversion.enable value.message_format_version (string). message.format.version value.message_timestamp_difference_max_ms (integer). message.timestamp.difference.max.ms value.message_timestamp_type (string). message.timestamp.type value.min_cleanable_dirty_ratio (number). min.cleanable.dirty.ratio value.min_compaction_lag_ms (integer). min.compaction.lag.ms value.min_insync_replicas (integer). min.insync.replicas value.preallocate (boolean). preallocate value.remote_storage_enable (boolean). remote_storage_enable.retention_bytes (integer). retention.bytes value.retention_ms (integer). retention.ms value.segment_bytes (integer). segment.bytes value.segment_index_bytes (integer). segment.index.bytes value.segment_jitter_ms (integer). segment.jitter.ms value.segment_ms (integer). segment.ms value.Appears on spec.
Kafka topic tags.
Required
key (string, Pattern: ^[a-zA-Z0-9_-]+$, MinLength: 1, MaxLength: 64).Optional
value (string, Pattern: ^[a-zA-Z0-9_-]+$, MaxLength: 256).apiVersion: aiven.io/v1alpha1\nkind: MySQL\nmetadata:\n name: my-mysql\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n connInfoSecretTarget:\n name: mysql-secret\n prefix: MY_SECRET_PREFIX_\n annotations:\n foo: bar\n labels:\n baz: egg\n\n project: my-aiven-project\n cloudName: google-europe-west1\n plan: business-4\n\n maintenanceWindowDow: sunday\n maintenanceWindowTime: 11:00:00\n\n userConfig:\n backup_hour: 17\n backup_minute: 11\n ip_filter:\n - network: 0.0.0.0\n description: whatever\n - network: 10.20.0.0/16\nInfo
To create this resource, a Secret containing Aiven token must be created first.
Apply the resource with:
kubectl apply -f example.yaml\nVerify the newly created MySQL:
kubectl get mysqls my-mysql\nThe output is similar to the following:
Name Project Region Plan State \nmy-mysql my-aiven-project google-europe-west1 business-4 RUNNING \nTo view the details of the Secret, use the following command:
kubectl describe secret mysql-secret\nYou can use the jq to quickly decode the Secret:
kubectl get secret mysql-secret -o json | jq '.data | map_values(@base64d)'\nThe output is similar to the following:
{\n \"MYSQL_HOST\": \"<secret>\",\n \"MYSQL_PORT\": \"<secret>\",\n \"MYSQL_DATABASE\": \"<secret>\",\n \"MYSQL_USER\": \"<secret>\",\n \"MYSQL_PASSWORD\": \"<secret>\",\n \"MYSQL_SSL_MODE\": \"<secret>\",\n \"MYSQL_URI\": \"<secret>\",\n \"MYSQL_REPLICA_URI\": \"<secret>\",\n \"MYSQL_CA_CERT\": \"<secret>\",\n}\nMySQL is the Schema for the mysqls API.
Exposes secret keys
MYSQL_HOST, MYSQL_PORT, MYSQL_DATABASE, MYSQL_USER, MYSQL_PASSWORD, MYSQL_SSL_MODE, MYSQL_URI, MYSQL_REPLICA_URI, MYSQL_CA_CERT.
Required
apiVersion (string). Value aiven.io/v1alpha1.kind (string). Value MySQL.metadata (object). Data that identifies the object, including a name string and optional namespace.spec (object). MySQLSpec defines the desired state of MySQL. See below for nested schema.Appears on MySQL.
MySQLSpec defines the desired state of MySQL.
Required
plan (string, MaxLength: 128). Subscription plan.project (string, Immutable, Pattern: ^[a-zA-Z0-9_-]+$, MaxLength: 63). Identifies the project this resource belongs to.Optional
authSecretRef (object). Authentication reference to Aiven token in a secret. See below for nested schema.cloudName (string, MaxLength: 256). Cloud the service runs in.connInfoSecretTarget (object). Secret configuration. See below for nested schema.connInfoSecretTargetDisabled (boolean, Immutable). When true, the secret containing connection information will not be created, defaults to false. This field cannot be changed after resource creation.disk_space (string, Pattern: (?i)^[1-9][0-9]*(GiB|G)?$). The disk space of the service, possible values depend on the service type, the cloud provider and the project. Reducing will result in the service re-balancing. The removal of this field does not change the value.maintenanceWindowDow (string, Enum: monday, tuesday, wednesday, thursday, friday, saturday, sunday). Day of week when maintenance operations should be performed. One monday, tuesday, wednesday, etc.maintenanceWindowTime (string, MaxLength: 8). Time of day when maintenance operations should be performed. UTC time in HH:mm:ss format.projectVPCRef (object). ProjectVPCRef reference to ProjectVPC resource to use its ID as ProjectVPCID automatically. See below for nested schema.projectVpcId (string, MaxLength: 36). Identifier of the VPC the service should be in, if any.serviceIntegrations (array of objects, Immutable, MaxItems: 1). Service integrations to specify when creating a service. Not applied after initial service creation. See below for nested schema.tags (object, AdditionalProperties: string). Tags are key-value pairs that allow you to categorize services.technicalEmails (array of objects, MaxItems: 10). Defines the email addresses that will receive alerts about upcoming maintenance updates or warnings about service instability. See below for nested schema.terminationProtection (boolean). Prevent service from being deleted. It is recommended to have this enabled for all services.userConfig (object). MySQL specific user configuration options. See below for nested schema.Appears on spec.
Authentication reference to Aiven token in a secret.
Required
key (string, MinLength: 1).name (string, MinLength: 1).Appears on spec.
Secret configuration.
Required
name (string, Immutable). Name of the secret resource to be created. By default, it is equal to the resource name.Optional
annotations (object, AdditionalProperties: string). Annotations added to the secret.labels (object, AdditionalProperties: string). Labels added to the secret.prefix (string). Prefix for the secret's keys. Added \"as is\" without any transformations. By default, is equal to the kind name in uppercase + underscore, e.g. KAFKA_, REDIS_, etc.Appears on spec.
ProjectVPCRef reference to ProjectVPC resource to use its ID as ProjectVPCID automatically.
Required
name (string, MinLength: 1).Optional
namespace (string, MinLength: 1).Appears on spec.
Service integrations to specify when creating a service. Not applied after initial service creation.
Required
integrationType (string, Enum: read_replica).sourceServiceName (string, MinLength: 1, MaxLength: 64).Appears on spec.
Defines the email addresses that will receive alerts about upcoming maintenance updates or warnings about service instability.
Required
email (string). Email address.Appears on spec.
MySQL specific user configuration options.
Optional
additional_backup_regions (array of strings, MaxItems: 1). Additional Cloud Regions for Backup Replication.admin_password (string, Immutable, Pattern: ^[a-zA-Z0-9-_]+$, MinLength: 8, MaxLength: 256). Custom password for admin user. Defaults to random string. This must be set only when a new service is being created.admin_username (string, Immutable, Pattern: ^[_A-Za-z0-9][-._A-Za-z0-9]{0,63}$, MaxLength: 64). Custom username for admin user. This must be set only when a new service is being created.backup_hour (integer, Minimum: 0, Maximum: 23). The hour of day (in UTC) when backup for the service is started. New backup is only started if previous backup has already completed.backup_minute (integer, Minimum: 0, Maximum: 59). The minute of an hour when backup for the service is started. New backup is only started if previous backup has already completed.binlog_retention_period (integer, Minimum: 600, Maximum: 86400). The minimum amount of time in seconds to keep binlog entries before deletion. This may be extended for services that require binlog entries for longer than the default for example if using the MySQL Debezium Kafka connector.ip_filter (array of objects, MaxItems: 1024). Allow incoming connections from CIDR address block, e.g. 10.20.0.0/16. See below for nested schema.migration (object). Migrate data from existing server. See below for nested schema.mysql (object). mysql.conf configuration values. See below for nested schema.mysql_version (string, Enum: 8). MySQL major version.private_access (object). Allow access to selected service ports from private networks. See below for nested schema.privatelink_access (object). Allow access to selected service components through Privatelink. See below for nested schema.project_to_fork_from (string, Immutable, Pattern: ^[a-z][-a-z0-9]{0,63}$|^$, MaxLength: 63). Name of another project to fork a service from. This has effect only when a new service is being created.public_access (object). Allow access to selected service ports from the public Internet. See below for nested schema.recovery_target_time (string, Immutable, MaxLength: 32). Recovery target time when forking a service. This has effect only when a new service is being created.service_log (boolean). Store logs for the service so that they are available in the HTTP API and console.service_to_fork_from (string, Immutable, Pattern: ^[a-z][-a-z0-9]{0,63}$|^$, MaxLength: 64). Name of another service to fork from. This has effect only when a new service is being created.static_ips (boolean). Use static public IP addresses.Appears on spec.userConfig.
Allow incoming connections from CIDR address block, e.g. 10.20.0.0/16.
Required
network (string, MaxLength: 43). CIDR address block.Optional
description (string, MaxLength: 1024). Description for IP filter list entry.Appears on spec.userConfig.
Migrate data from existing server.
Required
host (string, MaxLength: 255). Hostname or IP address of the server where to migrate data from.port (integer, Minimum: 1, Maximum: 65535). Port number of the server where to migrate data from.Optional
dbname (string, MaxLength: 63). Database name for bootstrapping the initial connection.ignore_dbs (string, MaxLength: 2048). Comma-separated list of databases, which should be ignored during migration (supported by MySQL and PostgreSQL only at the moment).method (string, Enum: dump, replication). The migration method to be used (currently supported only by Redis, Dragonfly, MySQL and PostgreSQL service types).password (string, MaxLength: 256). Password for authentication with the server where to migrate data from.ssl (boolean). The server where to migrate data from is secured with SSL.username (string, MaxLength: 256). User name for authentication with the server where to migrate data from.Appears on spec.userConfig.
mysql.conf configuration values.
Optional
connect_timeout (integer, Minimum: 2, Maximum: 3600). The number of seconds that the mysqld server waits for a connect packet before responding with Bad handshake.default_time_zone (string, Pattern: ^([-+][\\d:]*|[\\w/]*)$, MinLength: 2, MaxLength: 100). Default server time zone as an offset from UTC (from -12:00 to +12:00), a time zone name, or SYSTEM to use the MySQL server default.group_concat_max_len (integer, Minimum: 4). The maximum permitted result length in bytes for the GROUP_CONCAT() function.information_schema_stats_expiry (integer, Minimum: 900, Maximum: 31536000). The time, in seconds, before cached statistics expire.innodb_change_buffer_max_size (integer, Minimum: 0, Maximum: 50). Maximum size for the InnoDB change buffer, as a percentage of the total size of the buffer pool. Default is 25.innodb_flush_neighbors (integer, Minimum: 0, Maximum: 2). Specifies whether flushing a page from the InnoDB buffer pool also flushes other dirty pages in the same extent (default is 1): 0 - dirty pages in the same extent are not flushed, 1 - flush contiguous dirty pages in the same extent, 2 - flush dirty pages in the same extent.innodb_ft_min_token_size (integer, Minimum: 0, Maximum: 16). Minimum length of words that are stored in an InnoDB FULLTEXT index. Changing this parameter will lead to a restart of the MySQL service.innodb_ft_server_stopword_table (string, Pattern: ^.+/.+$, MaxLength: 1024). This option is used to specify your own InnoDB FULLTEXT index stopword list for all InnoDB tables.innodb_lock_wait_timeout (integer, Minimum: 1, Maximum: 3600). The length of time in seconds an InnoDB transaction waits for a row lock before giving up. Default is 120.innodb_log_buffer_size (integer, Minimum: 1048576, Maximum: 4294967295). The size in bytes of the buffer that InnoDB uses to write to the log files on disk.innodb_online_alter_log_max_size (integer, Minimum: 65536, Maximum: 1099511627776). The upper limit in bytes on the size of the temporary log files used during online DDL operations for InnoDB tables.innodb_print_all_deadlocks (boolean). When enabled, information about all deadlocks in InnoDB user transactions is recorded in the error log. Disabled by default.innodb_read_io_threads (integer, Minimum: 1, Maximum: 64). The number of I/O threads for read operations in InnoDB. Default is 4. Changing this parameter will lead to a restart of the MySQL service.innodb_rollback_on_timeout (boolean). When enabled a transaction timeout causes InnoDB to abort and roll back the entire transaction. Changing this parameter will lead to a restart of the MySQL service.innodb_thread_concurrency (integer, Minimum: 0, Maximum: 1000). Defines the maximum number of threads permitted inside of InnoDB. Default is 0 (infinite concurrency - no limit).innodb_write_io_threads (integer, Minimum: 1, Maximum: 64). The number of I/O threads for write operations in InnoDB. Default is 4. Changing this parameter will lead to a restart of the MySQL service.interactive_timeout (integer, Minimum: 30, Maximum: 604800). The number of seconds the server waits for activity on an interactive connection before closing it.internal_tmp_mem_storage_engine (string, Enum: TempTable, MEMORY). The storage engine for in-memory internal temporary tables.long_query_time (number, Minimum: 0, Maximum: 3600). The slow_query_logs work as SQL statements that take more than long_query_time seconds to execute. Default is 10s.max_allowed_packet (integer, Minimum: 102400, Maximum: 1073741824). Size of the largest message in bytes that can be received by the server. Default is 67108864 (64M).max_heap_table_size (integer, Minimum: 1048576, Maximum: 1073741824). Limits the size of internal in-memory tables. Also set tmp_table_size. Default is 16777216 (16M).net_buffer_length (integer, Minimum: 1024, Maximum: 1048576). Start sizes of connection buffer and result buffer. Default is 16384 (16K). Changing this parameter will lead to a restart of the MySQL service.net_read_timeout (integer, Minimum: 1, Maximum: 3600). The number of seconds to wait for more data from a connection before aborting the read.net_write_timeout (integer, Minimum: 1, Maximum: 3600). The number of seconds to wait for a block to be written to a connection before aborting the write.slow_query_log (boolean). Slow query log enables capturing of slow queries. Setting slow_query_log to false also truncates the mysql.slow_log table. Default is off.sort_buffer_size (integer, Minimum: 32768, Maximum: 1073741824). Sort buffer size in bytes for ORDER BY optimization. Default is 262144 (256K).sql_mode (string, Pattern: ^[A-Z_]*(,[A-Z_]+)*$, MaxLength: 1024). Global SQL mode. Set to empty to use MySQL server defaults. When creating a new service and not setting this field Aiven default SQL mode (strict, SQL standard compliant) will be assigned.sql_require_primary_key (boolean). Require primary key to be defined for new tables or old tables modified with ALTER TABLE and fail if missing. It is recommended to always have primary keys because various functionality may break if any large table is missing them.tmp_table_size (integer, Minimum: 1048576, Maximum: 1073741824). Limits the size of internal in-memory tables. Also set max_heap_table_size. Default is 16777216 (16M).wait_timeout (integer, Minimum: 1, Maximum: 2147483). The number of seconds the server waits for activity on a noninteractive connection before closing it.Appears on spec.userConfig.
Allow access to selected service ports from private networks.
Optional
mysql (boolean). Allow clients to connect to mysql with a DNS name that always resolves to the service's private IP addresses. Only available in certain network locations.mysqlx (boolean). Allow clients to connect to mysqlx with a DNS name that always resolves to the service's private IP addresses. Only available in certain network locations.prometheus (boolean). Allow clients to connect to prometheus with a DNS name that always resolves to the service's private IP addresses. Only available in certain network locations.Appears on spec.userConfig.
Allow access to selected service components through Privatelink.
Optional
mysql (boolean). Enable mysql.mysqlx (boolean). Enable mysqlx.prometheus (boolean). Enable prometheus.Appears on spec.userConfig.
Allow access to selected service ports from the public Internet.
Optional
mysql (boolean). Allow clients to connect to mysql from the public internet for service nodes that are in a project VPC or another type of private network.mysqlx (boolean). Allow clients to connect to mysqlx from the public internet for service nodes that are in a project VPC or another type of private network.prometheus (boolean). Allow clients to connect to prometheus from the public internet for service nodes that are in a project VPC or another type of private network.apiVersion: aiven.io/v1alpha1\nkind: OpenSearch\nmetadata:\n name: my-os\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n connInfoSecretTarget:\n name: os-secret\n prefix: MY_SECRET_PREFIX_\n annotations:\n foo: bar\n labels:\n baz: egg\n\n project: my-aiven-project\n cloudName: google-europe-west1\n plan: startup-4\n disk_space: 80GiB\n\n maintenanceWindowDow: friday\n maintenanceWindowTime: 23:00:00\nInfo
To create this resource, a Secret containing Aiven token must be created first.
Apply the resource with:
kubectl apply -f example.yaml\nVerify the newly created OpenSearch:
kubectl get opensearches my-os\nThe output is similar to the following:
Name Project Region Plan State \nmy-os my-aiven-project google-europe-west1 startup-4 RUNNING \nTo view the details of the Secret, use the following command:
kubectl describe secret os-secret\nYou can use the jq to quickly decode the Secret:
kubectl get secret os-secret -o json | jq '.data | map_values(@base64d)'\nThe output is similar to the following:
{\n \"OPENSEARCH_HOST\": \"<secret>\",\n \"OPENSEARCH_PORT\": \"<secret>\",\n \"OPENSEARCH_USER\": \"<secret>\",\n \"OPENSEARCH_PASSWORD\": \"<secret>\",\n}\nOpenSearch is the Schema for the opensearches API.
Exposes secret keys
OPENSEARCH_HOST, OPENSEARCH_PORT, OPENSEARCH_USER, OPENSEARCH_PASSWORD.
Required
apiVersion (string). Value aiven.io/v1alpha1.kind (string). Value OpenSearch.metadata (object). Data that identifies the object, including a name string and optional namespace.spec (object). OpenSearchSpec defines the desired state of OpenSearch. See below for nested schema.Appears on OpenSearch.
OpenSearchSpec defines the desired state of OpenSearch.
Required
plan (string, MaxLength: 128). Subscription plan.project (string, Immutable, Pattern: ^[a-zA-Z0-9_-]+$, MaxLength: 63). Identifies the project this resource belongs to.Optional
authSecretRef (object). Authentication reference to Aiven token in a secret. See below for nested schema.cloudName (string, MaxLength: 256). Cloud the service runs in.connInfoSecretTarget (object). Secret configuration. See below for nested schema.connInfoSecretTargetDisabled (boolean, Immutable). When true, the secret containing connection information will not be created, defaults to false. This field cannot be changed after resource creation.disk_space (string, Pattern: (?i)^[1-9][0-9]*(GiB|G)?$). The disk space of the service, possible values depend on the service type, the cloud provider and the project. Reducing will result in the service re-balancing. The removal of this field does not change the value.maintenanceWindowDow (string, Enum: monday, tuesday, wednesday, thursday, friday, saturday, sunday). Day of week when maintenance operations should be performed. One monday, tuesday, wednesday, etc.maintenanceWindowTime (string, MaxLength: 8). Time of day when maintenance operations should be performed. UTC time in HH:mm:ss format.projectVPCRef (object). ProjectVPCRef reference to ProjectVPC resource to use its ID as ProjectVPCID automatically. See below for nested schema.projectVpcId (string, MaxLength: 36). Identifier of the VPC the service should be in, if any.serviceIntegrations (array of objects, Immutable, MaxItems: 1). Service integrations to specify when creating a service. Not applied after initial service creation. See below for nested schema.tags (object, AdditionalProperties: string). Tags are key-value pairs that allow you to categorize services.technicalEmails (array of objects, MaxItems: 10). Defines the email addresses that will receive alerts about upcoming maintenance updates or warnings about service instability. See below for nested schema.terminationProtection (boolean). Prevent service from being deleted. It is recommended to have this enabled for all services.userConfig (object). OpenSearch specific user configuration options. See below for nested schema.Appears on spec.
Authentication reference to Aiven token in a secret.
Required
key (string, MinLength: 1).name (string, MinLength: 1).Appears on spec.
Secret configuration.
Required
name (string, Immutable). Name of the secret resource to be created. By default, it is equal to the resource name.Optional
annotations (object, AdditionalProperties: string). Annotations added to the secret.labels (object, AdditionalProperties: string). Labels added to the secret.prefix (string). Prefix for the secret's keys. Added \"as is\" without any transformations. By default, is equal to the kind name in uppercase + underscore, e.g. KAFKA_, REDIS_, etc.Appears on spec.
ProjectVPCRef reference to ProjectVPC resource to use its ID as ProjectVPCID automatically.
Required
name (string, MinLength: 1).Optional
namespace (string, MinLength: 1).Appears on spec.
Service integrations to specify when creating a service. Not applied after initial service creation.
Required
integrationType (string, Enum: read_replica).sourceServiceName (string, MinLength: 1, MaxLength: 64).Appears on spec.
Defines the email addresses that will receive alerts about upcoming maintenance updates or warnings about service instability.
Required
email (string). Email address.Appears on spec.
OpenSearch specific user configuration options.
Optional
additional_backup_regions (array of strings, MaxItems: 1). Additional Cloud Regions for Backup Replication.custom_domain (string, MaxLength: 255). Serve the web frontend using a custom CNAME pointing to the Aiven DNS name.disable_replication_factor_adjustment (boolean). DEPRECATED: Disable automatic replication factor adjustment for multi-node services. By default, Aiven ensures all indexes are replicated at least to two nodes. Note: Due to potential data loss in case of losing a service node, this setting can no longer be activated.index_patterns (array of objects, MaxItems: 512). Index patterns. See below for nested schema.index_template (object). Template settings for all new indexes. See below for nested schema.ip_filter (array of objects, MaxItems: 1024). Allow incoming connections from CIDR address block, e.g. 10.20.0.0/16. See below for nested schema.keep_index_refresh_interval (boolean). Aiven automation resets index.refresh_interval to default value for every index to be sure that indices are always visible to search. If it doesn't fit your case, you can disable this by setting up this flag to true.max_index_count (integer, Minimum: 0). DEPRECATED: use index_patterns instead.openid (object). OpenSearch OpenID Connect Configuration. See below for nested schema.opensearch (object). OpenSearch settings. See below for nested schema.opensearch_dashboards (object). OpenSearch Dashboards settings. See below for nested schema.opensearch_version (string, Enum: 1, 2). OpenSearch major version.private_access (object). Allow access to selected service ports from private networks. See below for nested schema.privatelink_access (object). Allow access to selected service components through Privatelink. See below for nested schema.project_to_fork_from (string, Immutable, Pattern: ^[a-z][-a-z0-9]{0,63}$|^$, MaxLength: 63). Name of another project to fork a service from. This has effect only when a new service is being created.public_access (object). Allow access to selected service ports from the public Internet. See below for nested schema.recovery_basebackup_name (string, Pattern: ^[a-zA-Z0-9-_:.]+$, MaxLength: 128). Name of the basebackup to restore in forked service.saml (object). OpenSearch SAML configuration. See below for nested schema.service_log (boolean). Store logs for the service so that they are available in the HTTP API and console.service_to_fork_from (string, Immutable, Pattern: ^[a-z][-a-z0-9]{0,63}$|^$, MaxLength: 64). Name of another service to fork from. This has effect only when a new service is being created.static_ips (boolean). Use static public IP addresses.Appears on spec.userConfig.
Index patterns.
Required
max_index_count (integer, Minimum: 0). Maximum number of indexes to keep.pattern (string, Pattern: ^[A-Za-z0-9-_.*?]+$, MaxLength: 1024). fnmatch pattern.Optional
sorting_algorithm (string, Enum: alphabetical, creation_date). Deletion sorting algorithm.Appears on spec.userConfig.
Template settings for all new indexes.
Optional
mapping_nested_objects_limit (integer, Minimum: 0, Maximum: 100000). The maximum number of nested JSON objects that a single document can contain across all nested types. This limit helps to prevent out of memory errors when a document contains too many nested objects. Default is 10000.number_of_replicas (integer, Minimum: 0, Maximum: 29). The number of replicas each primary shard has.number_of_shards (integer, Minimum: 1, Maximum: 1024). The number of primary shards that an index should have.Appears on spec.userConfig.
Allow incoming connections from CIDR address block, e.g. 10.20.0.0/16.
Required
network (string, MaxLength: 43). CIDR address block.Optional
description (string, MaxLength: 1024). Description for IP filter list entry.Appears on spec.userConfig.
OpenSearch OpenID Connect Configuration.
Required
client_id (string, Pattern: ^[^\\r\\n]*$, MinLength: 1, MaxLength: 1024). The ID of the OpenID Connect client configured in your IdP. Required.client_secret (string, Pattern: ^[^\\r\\n]*$, MinLength: 1, MaxLength: 1024). The client secret of the OpenID Connect client configured in your IdP. Required.connect_url (string, MaxLength: 2048). The URL of your IdP where the Security plugin can find the OpenID Connect metadata/configuration settings.enabled (boolean). Enables or disables OpenID Connect authentication for OpenSearch. When enabled, users can authenticate using OpenID Connect with an Identity Provider.Optional
header (string, Pattern: ^[^\\r\\n]*$, MinLength: 1, MaxLength: 1024). HTTP header name of the JWT token. Optional. Default is Authorization.jwt_header (string, Pattern: ^[^\\r\\n]*$, MinLength: 1, MaxLength: 1024). The HTTP header that stores the token. Typically the Authorization header with the Bearer schema: Authorization: Bearer . Optional. Default is Authorization. jwt_url_parameter (string, Pattern: ^[^\\r\\n]*$, MinLength: 1, MaxLength: 1024). If the token is not transmitted in the HTTP header, but as an URL parameter, define the name of the parameter here. Optional.refresh_rate_limit_count (integer, Minimum: 10). The maximum number of unknown key IDs in the time frame. Default is 10. Optional.refresh_rate_limit_time_window_ms (integer, Minimum: 10000). The time frame to use when checking the maximum number of unknown key IDs, in milliseconds. Optional.Default is 10000 (10 seconds).roles_key (string, Pattern: ^[^\\r\\n]*$, MinLength: 1, MaxLength: 1024). The key in the JSON payload that stores the user\u2019s roles. The value of this key must be a comma-separated list of roles. Required only if you want to use roles in the JWT.scope (string, Pattern: ^[^\\r\\n]*$, MinLength: 1, MaxLength: 1024). The scope of the identity token issued by the IdP. Optional. Default is openid profile email address phone.subject_key (string, Pattern: ^[^\\r\\n]*$, MinLength: 1, MaxLength: 1024). The key in the JSON payload that stores the user\u2019s name. If not defined, the subject registered claim is used. Most IdP providers use the preferred_username claim. Optional.Appears on spec.userConfig.
OpenSearch settings.
Optional
action_auto_create_index_enabled (boolean). Explicitly allow or block automatic creation of indices. Defaults to true.action_destructive_requires_name (boolean). Require explicit index names when deleting.auth_failure_listeners (object). Opensearch Security Plugin Settings. See below for nested schema.cluster_max_shards_per_node (integer, Minimum: 100, Maximum: 10000). Controls the number of shards allowed in the cluster per data node.cluster_routing_allocation_node_concurrent_recoveries (integer, Minimum: 2, Maximum: 16). How many concurrent incoming/outgoing shard recoveries (normally replicas) are allowed to happen on a node. Defaults to 2.email_sender_name (string, Pattern: ^[a-zA-Z0-9-_]+$, MaxLength: 40). Sender name placeholder to be used in Opensearch Dashboards and Opensearch keystore.email_sender_password (string, Pattern: ^[^\\x00-\\x1F]+$, MaxLength: 1024). Sender password for Opensearch alerts to authenticate with SMTP server.email_sender_username (string, Pattern: ^[^\\x00-\\x1F]+$, MaxLength: 320). Sender username for Opensearch alerts.enable_security_audit (boolean). Enable/Disable security audit.http_max_content_length (integer, Minimum: 1, Maximum: 2147483647). Maximum content length for HTTP requests to the OpenSearch HTTP API, in bytes.http_max_header_size (integer, Minimum: 1024, Maximum: 262144). The max size of allowed headers, in bytes.http_max_initial_line_length (integer, Minimum: 1024, Maximum: 65536). The max length of an HTTP URL, in bytes.indices_fielddata_cache_size (integer, Minimum: 3, Maximum: 100). Relative amount. Maximum amount of heap memory used for field data cache. This is an expert setting; decreasing the value too much will increase overhead of loading field data; too much memory used for field data cache will decrease amount of heap available for other operations.indices_memory_index_buffer_size (integer, Minimum: 3, Maximum: 40). Percentage value. Default is 10%. Total amount of heap used for indexing buffer, before writing segments to disk. This is an expert setting. Too low value will slow down indexing; too high value will increase indexing performance but causes performance issues for query performance.indices_memory_max_index_buffer_size (integer, Minimum: 3, Maximum: 2048). Absolute value. Default is unbound. Doesn't work without indices.memory.index_buffer_size. Maximum amount of heap used for query cache, an absolute indices.memory.index_buffer_size maximum hard limit.indices_memory_min_index_buffer_size (integer, Minimum: 3, Maximum: 2048). Absolute value. Default is 48mb. Doesn't work without indices.memory.index_buffer_size. Minimum amount of heap used for query cache, an absolute indices.memory.index_buffer_size minimal hard limit.indices_queries_cache_size (integer, Minimum: 3, Maximum: 40). Percentage value. Default is 10%. Maximum amount of heap used for query cache. This is an expert setting. Too low value will decrease query performance and increase performance for other operations; too high value will cause issues with other OpenSearch functionality.indices_query_bool_max_clause_count (integer, Minimum: 64, Maximum: 4096). Maximum number of clauses Lucene BooleanQuery can have. The default value (1024) is relatively high, and increasing it may cause performance issues. Investigate other approaches first before increasing this value.indices_recovery_max_bytes_per_sec (integer, Minimum: 40, Maximum: 400). Limits total inbound and outbound recovery traffic for each node. Applies to both peer recoveries as well as snapshot recoveries (i.e., restores from a snapshot). Defaults to 40mb.indices_recovery_max_concurrent_file_chunks (integer, Minimum: 2, Maximum: 5). Number of file chunks sent in parallel for each recovery. Defaults to 2.ism_enabled (boolean). Specifies whether ISM is enabled or not.ism_history_enabled (boolean). Specifies whether audit history is enabled or not. The logs from ISM are automatically indexed to a logs document.ism_history_max_age (integer, Minimum: 1, Maximum: 2147483647). The maximum age before rolling over the audit history index in hours.ism_history_max_docs (integer, Minimum: 1). The maximum number of documents before rolling over the audit history index.ism_history_rollover_check_period (integer, Minimum: 1, Maximum: 2147483647). The time between rollover checks for the audit history index in hours.ism_history_rollover_retention_period (integer, Minimum: 1, Maximum: 2147483647). How long audit history indices are kept in days.override_main_response_version (boolean). Compatibility mode sets OpenSearch to report its version as 7.10 so clients continue to work. Default is false.plugins_alerting_filter_by_backend_roles (boolean). Enable or disable filtering of alerting by backend roles. Requires Security plugin. Defaults to false.reindex_remote_whitelist (array of strings, MaxItems: 32). Whitelisted addresses for reindexing. Changing this value will cause all OpenSearch instances to restart.script_max_compilations_rate (string, MaxLength: 1024). Script compilation circuit breaker limits the number of inline script compilations within a period of time. Default is use-context.search_max_buckets (integer, Minimum: 1, Maximum: 1000000). Maximum number of aggregation buckets allowed in a single response. OpenSearch default value is used when this is not defined.thread_pool_analyze_queue_size (integer, Minimum: 10, Maximum: 2000). Size for the thread pool queue. See documentation for exact details.thread_pool_analyze_size (integer, Minimum: 1, Maximum: 128). Size for the thread pool. See documentation for exact details. Do note this may have maximum value depending on CPU count - value is automatically lowered if set to higher than maximum value.thread_pool_force_merge_size (integer, Minimum: 1, Maximum: 128). Size for the thread pool. See documentation for exact details. Do note this may have maximum value depending on CPU count - value is automatically lowered if set to higher than maximum value.thread_pool_get_queue_size (integer, Minimum: 10, Maximum: 2000). Size for the thread pool queue. See documentation for exact details.thread_pool_get_size (integer, Minimum: 1, Maximum: 128). Size for the thread pool. See documentation for exact details. Do note this may have maximum value depending on CPU count - value is automatically lowered if set to higher than maximum value.thread_pool_search_queue_size (integer, Minimum: 10, Maximum: 2000). Size for the thread pool queue. See documentation for exact details.thread_pool_search_size (integer, Minimum: 1, Maximum: 128). Size for the thread pool. See documentation for exact details. Do note this may have maximum value depending on CPU count - value is automatically lowered if set to higher than maximum value.thread_pool_search_throttled_queue_size (integer, Minimum: 10, Maximum: 2000). Size for the thread pool queue. See documentation for exact details.thread_pool_search_throttled_size (integer, Minimum: 1, Maximum: 128). Size for the thread pool. See documentation for exact details. Do note this may have maximum value depending on CPU count - value is automatically lowered if set to higher than maximum value.thread_pool_write_queue_size (integer, Minimum: 10, Maximum: 2000). Size for the thread pool queue. See documentation for exact details.thread_pool_write_size (integer, Minimum: 1, Maximum: 128). Size for the thread pool. See documentation for exact details. Do note this may have maximum value depending on CPU count - value is automatically lowered if set to higher than maximum value.Appears on spec.userConfig.opensearch.
Opensearch Security Plugin Settings.
Optional
internal_authentication_backend_limiting (object). See below for nested schema.ip_rate_limiting (object). IP address rate limiting settings. See below for nested schema.Appears on spec.userConfig.opensearch.auth_failure_listeners.
Optional
allowed_tries (integer, Minimum: 0, Maximum: 2147483647). The number of login attempts allowed before login is blocked.authentication_backend (string, Enum: internal, MaxLength: 1024). internal_authentication_backend_limiting.authentication_backend.block_expiry_seconds (integer, Minimum: 0, Maximum: 2147483647). The duration of time that login remains blocked after a failed login.max_blocked_clients (integer, Minimum: 0, Maximum: 2147483647). internal_authentication_backend_limiting.max_blocked_clients.max_tracked_clients (integer, Minimum: 0, Maximum: 2147483647). The maximum number of tracked IP addresses that have failed login.time_window_seconds (integer, Minimum: 0, Maximum: 2147483647). The window of time in which the value for allowed_tries is enforced.type (string, Enum: username, MaxLength: 1024). internal_authentication_backend_limiting.type.Appears on spec.userConfig.opensearch.auth_failure_listeners.
IP address rate limiting settings.
Optional
allowed_tries (integer, Minimum: 1, Maximum: 2147483647). The number of login attempts allowed before login is blocked.block_expiry_seconds (integer, Minimum: 1, Maximum: 36000). The duration of time that login remains blocked after a failed login.max_blocked_clients (integer, Minimum: 0, Maximum: 2147483647). The maximum number of blocked IP addresses.max_tracked_clients (integer, Minimum: 0, Maximum: 2147483647). The maximum number of tracked IP addresses that have failed login.time_window_seconds (integer, Minimum: 1, Maximum: 36000). The window of time in which the value for allowed_tries is enforced.type (string, Enum: ip, MaxLength: 1024). The type of rate limiting.Appears on spec.userConfig.
OpenSearch Dashboards settings.
Optional
enabled (boolean). Enable or disable OpenSearch Dashboards.max_old_space_size (integer, Minimum: 64, Maximum: 2048). Limits the maximum amount of memory (in MiB) the OpenSearch Dashboards process can use. This sets the max_old_space_size option of the nodejs running the OpenSearch Dashboards. Note: the memory reserved by OpenSearch Dashboards is not available for OpenSearch.opensearch_request_timeout (integer, Minimum: 5000, Maximum: 120000). Timeout in milliseconds for requests made by OpenSearch Dashboards towards OpenSearch.Appears on spec.userConfig.
Allow access to selected service ports from private networks.
Optional
opensearch (boolean). Allow clients to connect to opensearch with a DNS name that always resolves to the service's private IP addresses. Only available in certain network locations.opensearch_dashboards (boolean). Allow clients to connect to opensearch_dashboards with a DNS name that always resolves to the service's private IP addresses. Only available in certain network locations.prometheus (boolean). Allow clients to connect to prometheus with a DNS name that always resolves to the service's private IP addresses. Only available in certain network locations.Appears on spec.userConfig.
Allow access to selected service components through Privatelink.
Optional
opensearch (boolean). Enable opensearch.opensearch_dashboards (boolean). Enable opensearch_dashboards.prometheus (boolean). Enable prometheus.Appears on spec.userConfig.
Allow access to selected service ports from the public Internet.
Optional
opensearch (boolean). Allow clients to connect to opensearch from the public internet for service nodes that are in a project VPC or another type of private network.opensearch_dashboards (boolean). Allow clients to connect to opensearch_dashboards from the public internet for service nodes that are in a project VPC or another type of private network.prometheus (boolean). Allow clients to connect to prometheus from the public internet for service nodes that are in a project VPC or another type of private network.Appears on spec.userConfig.
OpenSearch SAML configuration.
Required
enabled (boolean). Enables or disables SAML-based authentication for OpenSearch. When enabled, users can authenticate using SAML with an Identity Provider.idp_entity_id (string, Pattern: ^[^\\r\\n]*$, MinLength: 1, MaxLength: 1024). The unique identifier for the Identity Provider (IdP) entity that is used for SAML authentication. This value is typically provided by the IdP.idp_metadata_url (string, MinLength: 1, MaxLength: 2048). The URL of the SAML metadata for the Identity Provider (IdP). This is used to configure SAML-based authentication with the IdP.sp_entity_id (string, Pattern: ^[^\\r\\n]*$, MinLength: 1, MaxLength: 1024). The unique identifier for the Service Provider (SP) entity that is used for SAML authentication. This value is typically provided by the SP.Optional
idp_pemtrustedcas_content (string, MaxLength: 16384). This parameter specifies the PEM-encoded root certificate authority (CA) content for the SAML identity provider (IdP) server verification. The root CA content is used to verify the SSL/TLS certificate presented by the server.roles_key (string, Pattern: ^[^\\r\\n]*$, MinLength: 1, MaxLength: 256). Optional. Specifies the attribute in the SAML response where role information is stored, if available. Role attributes are not required for SAML authentication, but can be included in SAML assertions by most Identity Providers (IdPs) to determine user access levels or permissions.subject_key (string, Pattern: ^[^\\r\\n]*$, MinLength: 1, MaxLength: 256). Optional. Specifies the attribute in the SAML response where the subject identifier is stored. If not configured, the NameID attribute is used by default.apiVersion: aiven.io/v1alpha1\nkind: PostgreSQL\nmetadata:\n name: my-postgresql\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n connInfoSecretTarget:\n name: postgresql-secret\n prefix: MY_SECRET_PREFIX_\n annotations:\n foo: bar\n labels:\n baz: egg\n\n project: aiven-project-name\n cloudName: google-europe-west1\n plan: startup-4\n\n maintenanceWindowDow: sunday\n maintenanceWindowTime: 11:00:00\n\n userConfig:\n pg_version: \"15\"\nInfo
To create this resource, a Secret containing Aiven token must be created first.
Apply the resource with:
kubectl apply -f example.yaml\nVerify the newly created PostgreSQL:
kubectl get postgresqls my-postgresql\nThe output is similar to the following:
Name Project Region Plan State \nmy-postgresql aiven-project-name google-europe-west1 startup-4 RUNNING \nTo view the details of the Secret, use the following command:
kubectl describe secret postgresql-secret\nYou can use the jq to quickly decode the Secret:
kubectl get secret postgresql-secret -o json | jq '.data | map_values(@base64d)'\nThe output is similar to the following:
{\n \"POSTGRESQL_HOST\": \"<secret>\",\n \"POSTGRESQL_PORT\": \"<secret>\",\n \"POSTGRESQL_DATABASE\": \"<secret>\",\n \"POSTGRESQL_USER\": \"<secret>\",\n \"POSTGRESQL_PASSWORD\": \"<secret>\",\n \"POSTGRESQL_SSLMODE\": \"<secret>\",\n \"POSTGRESQL_DATABASE_URI\": \"<secret>\",\n \"POSTGRESQL_CA_CERT\": \"<secret>\",\n}\nPostgreSQL is the Schema for the postgresql API.
Exposes secret keys
POSTGRESQL_HOST, POSTGRESQL_PORT, POSTGRESQL_DATABASE, POSTGRESQL_USER, POSTGRESQL_PASSWORD, POSTGRESQL_SSLMODE, POSTGRESQL_DATABASE_URI, POSTGRESQL_CA_CERT.
Required
apiVersion (string). Value aiven.io/v1alpha1.kind (string). Value PostgreSQL.metadata (object). Data that identifies the object, including a name string and optional namespace.spec (object). PostgreSQLSpec defines the desired state of postgres instance. See below for nested schema.Appears on PostgreSQL.
PostgreSQLSpec defines the desired state of postgres instance.
Required
plan (string, MaxLength: 128). Subscription plan.project (string, Immutable, Pattern: ^[a-zA-Z0-9_-]+$, MaxLength: 63). Identifies the project this resource belongs to.Optional
authSecretRef (object). Authentication reference to Aiven token in a secret. See below for nested schema.cloudName (string, MaxLength: 256). Cloud the service runs in.connInfoSecretTarget (object). Secret configuration. See below for nested schema.connInfoSecretTargetDisabled (boolean, Immutable). When true, the secret containing connection information will not be created, defaults to false. This field cannot be changed after resource creation.disk_space (string, Pattern: (?i)^[1-9][0-9]*(GiB|G)?$). The disk space of the service, possible values depend on the service type, the cloud provider and the project. Reducing will result in the service re-balancing. The removal of this field does not change the value.maintenanceWindowDow (string, Enum: monday, tuesday, wednesday, thursday, friday, saturday, sunday). Day of week when maintenance operations should be performed. One monday, tuesday, wednesday, etc.maintenanceWindowTime (string, MaxLength: 8). Time of day when maintenance operations should be performed. UTC time in HH:mm:ss format.projectVPCRef (object). ProjectVPCRef reference to ProjectVPC resource to use its ID as ProjectVPCID automatically. See below for nested schema.projectVpcId (string, MaxLength: 36). Identifier of the VPC the service should be in, if any.serviceIntegrations (array of objects, Immutable, MaxItems: 1). Service integrations to specify when creating a service. Not applied after initial service creation. See below for nested schema.tags (object, AdditionalProperties: string). Tags are key-value pairs that allow you to categorize services.technicalEmails (array of objects, MaxItems: 10). Defines the email addresses that will receive alerts about upcoming maintenance updates or warnings about service instability. See below for nested schema.terminationProtection (boolean). Prevent service from being deleted. It is recommended to have this enabled for all services.userConfig (object). PostgreSQL specific user configuration options. See below for nested schema.Appears on spec.
Authentication reference to Aiven token in a secret.
Required
key (string, MinLength: 1).name (string, MinLength: 1).Appears on spec.
Secret configuration.
Required
name (string, Immutable). Name of the secret resource to be created. By default, it is equal to the resource name.Optional
annotations (object, AdditionalProperties: string). Annotations added to the secret.labels (object, AdditionalProperties: string). Labels added to the secret.prefix (string). Prefix for the secret's keys. Added \"as is\" without any transformations. By default, is equal to the kind name in uppercase + underscore, e.g. KAFKA_, REDIS_, etc.Appears on spec.
ProjectVPCRef reference to ProjectVPC resource to use its ID as ProjectVPCID automatically.
Required
name (string, MinLength: 1).Optional
namespace (string, MinLength: 1).Appears on spec.
Service integrations to specify when creating a service. Not applied after initial service creation.
Required
integrationType (string, Enum: read_replica).sourceServiceName (string, MinLength: 1, MaxLength: 64).Appears on spec.
Defines the email addresses that will receive alerts about upcoming maintenance updates or warnings about service instability.
Required
email (string). Email address.Appears on spec.
PostgreSQL specific user configuration options.
Optional
additional_backup_regions (array of strings, MaxItems: 1). Additional Cloud Regions for Backup Replication.admin_password (string, Immutable, Pattern: ^[a-zA-Z0-9-_]+$, MinLength: 8, MaxLength: 256). Custom password for admin user. Defaults to random string. This must be set only when a new service is being created.admin_username (string, Immutable, Pattern: ^[_A-Za-z0-9][-._A-Za-z0-9]{0,63}$, MaxLength: 64). Custom username for admin user. This must be set only when a new service is being created.backup_hour (integer, Minimum: 0, Maximum: 23). The hour of day (in UTC) when backup for the service is started. New backup is only started if previous backup has already completed.backup_minute (integer, Minimum: 0, Maximum: 59). The minute of an hour when backup for the service is started. New backup is only started if previous backup has already completed.enable_ipv6 (boolean). Register AAAA DNS records for the service, and allow IPv6 packets to service ports.ip_filter (array of objects, MaxItems: 1024). Allow incoming connections from CIDR address block, e.g. 10.20.0.0/16. See below for nested schema.migration (object). Migrate data from existing server. See below for nested schema.pg (object). postgresql.conf configuration values. See below for nested schema.pg_qualstats (object). Deprecated. System-wide settings for the pg_qualstats extension. See below for nested schema.pg_read_replica (boolean). Should the service which is being forked be a read replica (deprecated, use read_replica service integration instead).pg_service_to_fork_from (string, Immutable, Pattern: ^[a-z][-a-z0-9]{0,63}$|^$, MaxLength: 64). Name of the PG Service from which to fork (deprecated, use service_to_fork_from). This has effect only when a new service is being created.pg_stat_monitor_enable (boolean). Enable the pg_stat_monitor extension. Enabling this extension will cause the cluster to be restarted.When this extension is enabled, pg_stat_statements results for utility commands are unreliable.pg_version (string, Enum: 12, 13, 14, 15, 16). PostgreSQL major version.pgaudit (object). Deprecated. System-wide settings for the pgaudit extension. See below for nested schema.pgbouncer (object). PGBouncer connection pooling settings. See below for nested schema.pglookout (object). System-wide settings for pglookout. See below for nested schema.private_access (object). Allow access to selected service ports from private networks. See below for nested schema.privatelink_access (object). Allow access to selected service components through Privatelink. See below for nested schema.project_to_fork_from (string, Immutable, Pattern: ^[a-z][-a-z0-9]{0,63}$|^$, MaxLength: 63). Name of another project to fork a service from. This has effect only when a new service is being created.public_access (object). Allow access to selected service ports from the public Internet. See below for nested schema.recovery_target_time (string, Immutable, MaxLength: 32). Recovery target time when forking a service. This has effect only when a new service is being created.service_log (boolean). Store logs for the service so that they are available in the HTTP API and console.service_to_fork_from (string, Immutable, Pattern: ^[a-z][-a-z0-9]{0,63}$|^$, MaxLength: 64). Name of another service to fork from. This has effect only when a new service is being created.shared_buffers_percentage (number, Minimum: 20, Maximum: 60). Percentage of total RAM that the database server uses for shared memory buffers. Valid range is 20-60 (float), which corresponds to 20% - 60%. This setting adjusts the shared_buffers configuration value.static_ips (boolean). Use static public IP addresses.synchronous_replication (string, Enum: quorum, off). Synchronous replication type. Note that the service plan also needs to support synchronous replication.timescaledb (object). System-wide settings for the timescaledb extension. See below for nested schema.variant (string, Enum: aiven, timescale). Variant of the PostgreSQL service, may affect the features that are exposed by default.work_mem (integer, Minimum: 1, Maximum: 1024). Sets the maximum amount of memory to be used by a query operation (such as a sort or hash table) before writing to temporary disk files, in MB. Default is 1MB + 0.075% of total RAM (up to 32MB).Appears on spec.userConfig.
Allow incoming connections from CIDR address block, e.g. 10.20.0.0/16.
Required
network (string, MaxLength: 43). CIDR address block.Optional
description (string, MaxLength: 1024). Description for IP filter list entry.Appears on spec.userConfig.
Migrate data from existing server.
Required
host (string, MaxLength: 255). Hostname or IP address of the server where to migrate data from.port (integer, Minimum: 1, Maximum: 65535). Port number of the server where to migrate data from.Optional
dbname (string, MaxLength: 63). Database name for bootstrapping the initial connection.ignore_dbs (string, MaxLength: 2048). Comma-separated list of databases, which should be ignored during migration (supported by MySQL and PostgreSQL only at the moment).method (string, Enum: dump, replication). The migration method to be used (currently supported only by Redis, Dragonfly, MySQL and PostgreSQL service types).password (string, MaxLength: 256). Password for authentication with the server where to migrate data from.ssl (boolean). The server where to migrate data from is secured with SSL.username (string, MaxLength: 256). User name for authentication with the server where to migrate data from.Appears on spec.userConfig.
postgresql.conf configuration values.
Optional
autovacuum_analyze_scale_factor (number, Minimum: 0, Maximum: 1). Specifies a fraction of the table size to add to autovacuum_analyze_threshold when deciding whether to trigger an ANALYZE. The default is 0.2 (20% of table size).autovacuum_analyze_threshold (integer, Minimum: 0, Maximum: 2147483647). Specifies the minimum number of inserted, updated or deleted tuples needed to trigger an ANALYZE in any one table. The default is 50 tuples.autovacuum_freeze_max_age (integer, Minimum: 200000000, Maximum: 1500000000). Specifies the maximum age (in transactions) that a table's pg_class.relfrozenxid field can attain before a VACUUM operation is forced to prevent transaction ID wraparound within the table. Note that the system will launch autovacuum processes to prevent wraparound even when autovacuum is otherwise disabled. This parameter will cause the server to be restarted.autovacuum_max_workers (integer, Minimum: 1, Maximum: 20). Specifies the maximum number of autovacuum processes (other than the autovacuum launcher) that may be running at any one time. The default is three. This parameter can only be set at server start.autovacuum_naptime (integer, Minimum: 1, Maximum: 86400). Specifies the minimum delay between autovacuum runs on any given database. The delay is measured in seconds, and the default is one minute.autovacuum_vacuum_cost_delay (integer, Minimum: -1, Maximum: 100). Specifies the cost delay value that will be used in automatic VACUUM operations. If -1 is specified, the regular vacuum_cost_delay value will be used. The default value is 20 milliseconds.autovacuum_vacuum_cost_limit (integer, Minimum: -1, Maximum: 10000). Specifies the cost limit value that will be used in automatic VACUUM operations. If -1 is specified (which is the default), the regular vacuum_cost_limit value will be used.autovacuum_vacuum_scale_factor (number, Minimum: 0, Maximum: 1). Specifies a fraction of the table size to add to autovacuum_vacuum_threshold when deciding whether to trigger a VACUUM. The default is 0.2 (20% of table size).autovacuum_vacuum_threshold (integer, Minimum: 0, Maximum: 2147483647). Specifies the minimum number of updated or deleted tuples needed to trigger a VACUUM in any one table. The default is 50 tuples.bgwriter_delay (integer, Minimum: 10, Maximum: 10000). Specifies the delay between activity rounds for the background writer in milliseconds. Default is 200.bgwriter_flush_after (integer, Minimum: 0, Maximum: 2048). Whenever more than bgwriter_flush_after bytes have been written by the background writer, attempt to force the OS to issue these writes to the underlying storage. Specified in kilobytes, default is 512. Setting of 0 disables forced writeback.bgwriter_lru_maxpages (integer, Minimum: 0, Maximum: 1073741823). In each round, no more than this many buffers will be written by the background writer. Setting this to zero disables background writing. Default is 100.bgwriter_lru_multiplier (number, Minimum: 0, Maximum: 10). The average recent need for new buffers is multiplied by bgwriter_lru_multiplier to arrive at an estimate of the number that will be needed during the next round, (up to bgwriter_lru_maxpages). 1.0 represents a \u201cjust in time\u201d policy of writing exactly the number of buffers predicted to be needed. Larger values provide some cushion against spikes in demand, while smaller values intentionally leave writes to be done by server processes. The default is 2.0.deadlock_timeout (integer, Minimum: 500, Maximum: 1800000). This is the amount of time, in milliseconds, to wait on a lock before checking to see if there is a deadlock condition.default_toast_compression (string, Enum: lz4, pglz). Specifies the default TOAST compression method for values of compressible columns (the default is lz4).idle_in_transaction_session_timeout (integer, Minimum: 0, Maximum: 604800000). Time out sessions with open transactions after this number of milliseconds.jit (boolean). Controls system-wide use of Just-in-Time Compilation (JIT).log_autovacuum_min_duration (integer, Minimum: -1, Maximum: 2147483647). Causes each action executed by autovacuum to be logged if it ran for at least the specified number of milliseconds. Setting this to zero logs all autovacuum actions. Minus-one (the default) disables logging autovacuum actions.log_error_verbosity (string, Enum: TERSE, DEFAULT, VERBOSE). Controls the amount of detail written in the server log for each message that is logged.log_line_prefix (string, Enum: 'pid=%p,user=%u,db=%d,app=%a,client=%h ', '%t [%p]: [%l-1] user=%u,db=%d,app=%a,client=%h ', '%m [%p] %q[user=%u,db=%d,app=%a] '). Choose from one of the available log formats.log_min_duration_statement (integer, Minimum: -1, Maximum: 86400000). Log statements that take more than this number of milliseconds to run, -1 disables.log_temp_files (integer, Minimum: -1, Maximum: 2147483647). Log statements for each temporary file created larger than this number of kilobytes, -1 disables.max_files_per_process (integer, Minimum: 1000, Maximum: 4096). PostgreSQL maximum number of files that can be open per process.max_locks_per_transaction (integer, Minimum: 64, Maximum: 6400). PostgreSQL maximum locks per transaction.max_logical_replication_workers (integer, Minimum: 4, Maximum: 64). PostgreSQL maximum logical replication workers (taken from the pool of max_parallel_workers).max_parallel_workers (integer, Minimum: 0, Maximum: 96). Sets the maximum number of workers that the system can support for parallel queries.max_parallel_workers_per_gather (integer, Minimum: 0, Maximum: 96). Sets the maximum number of workers that can be started by a single Gather or Gather Merge node.max_pred_locks_per_transaction (integer, Minimum: 64, Maximum: 5120). PostgreSQL maximum predicate locks per transaction.max_prepared_transactions (integer, Minimum: 0, Maximum: 10000). PostgreSQL maximum prepared transactions.max_replication_slots (integer, Minimum: 8, Maximum: 64). PostgreSQL maximum replication slots.max_slot_wal_keep_size (integer, Minimum: -1, Maximum: 2147483647). PostgreSQL maximum WAL size (MB) reserved for replication slots. Default is -1 (unlimited). wal_keep_size minimum WAL size setting takes precedence over this.max_stack_depth (integer, Minimum: 2097152, Maximum: 6291456). Maximum depth of the stack in bytes.max_standby_archive_delay (integer, Minimum: 1, Maximum: 43200000). Max standby archive delay in milliseconds.max_standby_streaming_delay (integer, Minimum: 1, Maximum: 43200000). Max standby streaming delay in milliseconds.max_wal_senders (integer, Minimum: 20, Maximum: 64). PostgreSQL maximum WAL senders.max_worker_processes (integer, Minimum: 8, Maximum: 96). Sets the maximum number of background processes that the system can support.pg_partman_bgw.interval (integer, Minimum: 3600, Maximum: 604800). Sets the time interval to run pg_partman's scheduled tasks.pg_partman_bgw.role (string, Pattern: ^[_A-Za-z0-9][-._A-Za-z0-9]{0,63}$, MaxLength: 64). Controls which role to use for pg_partman's scheduled background tasks.pg_stat_monitor.pgsm_enable_query_plan (boolean). Enables or disables query plan monitoring.pg_stat_monitor.pgsm_max_buckets (integer, Minimum: 1, Maximum: 10). Sets the maximum number of buckets.pg_stat_statements.track (string, Enum: all, top, none). Controls which statements are counted. Specify top to track top-level statements (those issued directly by clients), all to also track nested statements (such as statements invoked within functions), or none to disable statement statistics collection. The default value is top.temp_file_limit (integer, Minimum: -1, Maximum: 2147483647). PostgreSQL temporary file limit in KiB, -1 for unlimited.timezone (string, Pattern: ^[\\w/]*$, MaxLength: 64). PostgreSQL service timezone.track_activity_query_size (integer, Minimum: 1024, Maximum: 10240). Specifies the number of bytes reserved to track the currently executing command for each active session.track_commit_timestamp (string, Enum: off, on). Record commit time of transactions.track_functions (string, Enum: all, pl, none). Enables tracking of function call counts and time used.track_io_timing (string, Enum: off, on). Enables timing of database I/O calls. This parameter is off by default, because it will repeatedly query the operating system for the current time, which may cause significant overhead on some platforms.wal_sender_timeout (integer). Terminate replication connections that are inactive for longer than this amount of time, in milliseconds. Setting this value to zero disables the timeout.wal_writer_delay (integer, Minimum: 10, Maximum: 200). WAL flush interval in milliseconds. Note that setting this value to lower than the default 200ms may negatively impact performance.Appears on spec.userConfig.
Deprecated. System-wide settings for the pg_qualstats extension.
Optional
enabled (boolean). Deprecated. Enable / Disable pg_qualstats.min_err_estimate_num (integer, Minimum: 0). Deprecated. Error estimation num threshold to save quals.min_err_estimate_ratio (integer, Minimum: 0). Deprecated. Error estimation ratio threshold to save quals.track_constants (boolean). Deprecated. Enable / Disable pg_qualstats constants tracking.track_pg_catalog (boolean). Deprecated. Track quals on system catalogs too.Appears on spec.userConfig.
Deprecated. System-wide settings for the pgaudit extension.
Optional
feature_enabled (boolean). Deprecated. Enable pgaudit extension. When enabled, pgaudit extension will be automatically installed.Otherwise, extension will be uninstalled but auditing configurations will be preserved.log (array of strings). Deprecated. Specifies which classes of statements will be logged by session audit logging.log_catalog (boolean). Deprecated. Specifies that session logging should be enabled in the casewhere all relations in a statement are in pg_catalog.log_client (boolean). Deprecated. Specifies whether log messages will be visible to a client process such as psql.log_level (string). Deprecated. Specifies the log level that will be used for log entries.log_max_string_length (integer, Minimum: -1, Maximum: 102400). Deprecated. Crop parameters representation and whole statements if they exceed this threshold. A (default) value of -1 disable the truncation.log_nested_statements (boolean). Deprecated. This GUC allows to turn off logging nested statements, that is, statements that are executed as part of another ExecutorRun.log_parameter (boolean). Deprecated. Specifies that audit logging should include the parameters that were passed with the statement.log_parameter_max_size (integer). Deprecated. Specifies that parameter values longer than this setting (in bytes) should not be logged, but replaced with . log_relation (boolean). Deprecated. Specifies whether session audit logging should create a separate log entry for each relation (TABLE, VIEW, etc.) referenced in a SELECT or DML statement.log_rows (boolean). Deprecated. Specifies that audit logging should include the rows retrieved or affected by a statement. When enabled the rows field will be included after the parameter field.log_statement (boolean). Deprecated. Specifies whether logging will include the statement text and parameters (if enabled).log_statement_once (boolean). Deprecated. Specifies whether logging will include the statement text and parameters with the first log entry for a statement/substatement combination or with every entry.role (string, Pattern: ^[_A-Za-z0-9][-._A-Za-z0-9]{0,63}$, MaxLength: 64). Deprecated. Specifies the master role to use for object audit logging.Appears on spec.userConfig.
PGBouncer connection pooling settings.
Optional
autodb_idle_timeout (integer, Minimum: 0, Maximum: 86400). If the automatically created database pools have been unused this many seconds, they are freed. If 0 then timeout is disabled. [seconds].autodb_max_db_connections (integer, Minimum: 0, Maximum: 2147483647). Do not allow more than this many server connections per database (regardless of user). Setting it to 0 means unlimited.autodb_pool_mode (string, Enum: session, transaction, statement). PGBouncer pool mode.autodb_pool_size (integer, Minimum: 0, Maximum: 10000). If non-zero then create automatically a pool of that size per user when a pool doesn't exist.ignore_startup_parameters (array of strings, MaxItems: 32). List of parameters to ignore when given in startup packet.min_pool_size (integer, Minimum: 0, Maximum: 10000). Add more server connections to pool if below this number. Improves behavior when usual load comes suddenly back after period of total inactivity. The value is effectively capped at the pool size.server_idle_timeout (integer, Minimum: 0, Maximum: 86400). If a server connection has been idle more than this many seconds it will be dropped. If 0 then timeout is disabled. [seconds].server_lifetime (integer, Minimum: 60, Maximum: 86400). The pooler will close an unused server connection that has been connected longer than this. [seconds].server_reset_query_always (boolean). Run server_reset_query (DISCARD ALL) in all pooling modes.Appears on spec.userConfig.
System-wide settings for pglookout.
Required
max_failover_replication_time_lag (integer, Minimum: 10). Number of seconds of master unavailability before triggering database failover to standby.Appears on spec.userConfig.
Allow access to selected service ports from private networks.
Optional
pg (boolean). Allow clients to connect to pg with a DNS name that always resolves to the service's private IP addresses. Only available in certain network locations.pgbouncer (boolean). Allow clients to connect to pgbouncer with a DNS name that always resolves to the service's private IP addresses. Only available in certain network locations.prometheus (boolean). Allow clients to connect to prometheus with a DNS name that always resolves to the service's private IP addresses. Only available in certain network locations.Appears on spec.userConfig.
Allow access to selected service components through Privatelink.
Optional
pg (boolean). Enable pg.pgbouncer (boolean). Enable pgbouncer.prometheus (boolean). Enable prometheus.Appears on spec.userConfig.
Allow access to selected service ports from the public Internet.
Optional
pg (boolean). Allow clients to connect to pg from the public internet for service nodes that are in a project VPC or another type of private network.pgbouncer (boolean). Allow clients to connect to pgbouncer from the public internet for service nodes that are in a project VPC or another type of private network.prometheus (boolean). Allow clients to connect to prometheus from the public internet for service nodes that are in a project VPC or another type of private network.Appears on spec.userConfig.
System-wide settings for the timescaledb extension.
Required
max_background_workers (integer, Minimum: 1, Maximum: 4096). The number of background workers for timescaledb operations. You should configure this setting to the sum of your number of databases and the total number of concurrent background workers you want running at any given point in time.apiVersion: aiven.io/v1alpha1\nkind: Project\nmetadata:\n name: my-project\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n connInfoSecretTarget:\n name: project-secret\n prefix: MY_SECRET_PREFIX_\n annotations:\n foo: bar\n labels:\n baz: egg\n\n tags:\n env: prod\n\n billingAddress: NYC\n cloud: aws-eu-west-1\nInfo
To create this resource, a Secret containing Aiven token must be created first.
Apply the resource with:
kubectl apply -f example.yaml\nVerify the newly created Project:
kubectl get projects my-project\nThe output is similar to the following:
Name \nmy-project \nTo view the details of the Secret, use the following command:
kubectl describe secret project-secret\nYou can use the jq to quickly decode the Secret:
kubectl get secret project-secret -o json | jq '.data | map_values(@base64d)'\nThe output is similar to the following:
{\n \"PROJECT_CA_CERT\": \"<secret>\",\n}\nProject is the Schema for the projects API.
Exposes secret keys
PROJECT_CA_CERT.
Required
apiVersion (string). Value aiven.io/v1alpha1.kind (string). Value Project.metadata (object). Data that identifies the object, including a name string and optional namespace.spec (object). ProjectSpec defines the desired state of Project. See below for nested schema.Appears on Project.
ProjectSpec defines the desired state of Project.
Optional
accountId (string, MaxLength: 32). Account ID.authSecretRef (object). Authentication reference to Aiven token in a secret. See below for nested schema.billingAddress (string, MaxLength: 1000). Billing name and address of the project.billingCurrency (string, Enum: AUD, CAD, CHF, DKK, EUR, GBP, NOK, SEK, USD). Billing currency.billingEmails (array of strings, MaxItems: 10). Billing contact emails of the project.billingExtraText (string, MaxLength: 1000). Extra text to be included in all project invoices, e.g. purchase order or cost center number.billingGroupId (string, Immutable, MinLength: 36, MaxLength: 36). BillingGroup ID.cardId (string, MaxLength: 64). Credit card ID; The ID may be either last 4 digits of the card or the actual ID.cloud (string, MaxLength: 256). Target cloud, example: aws-eu-central-1.connInfoSecretTarget (object). Secret configuration. See below for nested schema.connInfoSecretTargetDisabled (boolean, Immutable). When true, the secret containing connection information will not be created, defaults to false. This field cannot be changed after resource creation.copyFromProject (string, Immutable, MaxLength: 63). Project name from which to copy settings to the new project.countryCode (string, MinLength: 2, MaxLength: 2). Billing country code of the project.tags (object, AdditionalProperties: string). Tags are key-value pairs that allow you to categorize projects.technicalEmails (array of strings, MaxItems: 10). Technical contact emails of the project.Appears on spec.
Authentication reference to Aiven token in a secret.
Required
key (string, MinLength: 1).name (string, MinLength: 1).Appears on spec.
Secret configuration.
Required
name (string, Immutable). Name of the secret resource to be created. By default, it is equal to the resource name.Optional
annotations (object, AdditionalProperties: string). Annotations added to the secret.labels (object, AdditionalProperties: string). Labels added to the secret.prefix (string). Prefix for the secret's keys. Added \"as is\" without any transformations. By default, is equal to the kind name in uppercase + underscore, e.g. KAFKA_, REDIS_, etc.apiVersion: aiven.io/v1alpha1\nkind: ProjectVPC\nmetadata:\n name: my-project-vpc\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n project: aiven-project-name\n cloudName: google-europe-west1\n networkCidr: 10.0.0.0/24\nInfo
To create this resource, a Secret containing Aiven token must be created first.
Apply the resource with:
kubectl apply -f example.yaml\nVerify the newly created ProjectVPC:
kubectl get projectvpcs my-project-vpc\nThe output is similar to the following:
Name Project Cloud Network CIDR State \nmy-project-vpc aiven-project-name google-europe-west1 10.0.0.0/24 RUNNING \nProjectVPC is the Schema for the projectvpcs API.
Required
apiVersion (string). Value aiven.io/v1alpha1.kind (string). Value ProjectVPC.metadata (object). Data that identifies the object, including a name string and optional namespace.spec (object). ProjectVPCSpec defines the desired state of ProjectVPC. See below for nested schema.Appears on ProjectVPC.
ProjectVPCSpec defines the desired state of ProjectVPC.
Required
cloudName (string, Immutable, MaxLength: 256). Cloud the VPC is in.networkCidr (string, Immutable, MaxLength: 36). Network address range used by the VPC like 192.168.0.0/24.project (string, Immutable, Pattern: ^[a-zA-Z0-9_-]+$, MaxLength: 63). Identifies the project this resource belongs to.Optional
authSecretRef (object). Authentication reference to Aiven token in a secret. See below for nested schema.Appears on spec.
Authentication reference to Aiven token in a secret.
Required
key (string, MinLength: 1).name (string, MinLength: 1).apiVersion: aiven.io/v1alpha1\nkind: Redis\nmetadata:\n name: k8s-redis\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n connInfoSecretTarget:\n name: redis-token\n prefix: MY_SECRET_PREFIX_\n annotations:\n foo: bar\n labels:\n baz: egg\n\n project: my-aiven-project\n cloudName: google-europe-west1\n plan: startup-4\n\n maintenanceWindowDow: friday\n maintenanceWindowTime: 23:00:00\n\n userConfig:\n redis_maxmemory_policy: allkeys-random\nInfo
To create this resource, a Secret containing Aiven token must be created first.
Apply the resource with:
kubectl apply -f example.yaml\nVerify the newly created Redis:
kubectl get redis k8s-redis\nThe output is similar to the following:
Name Project Region Plan State \nk8s-redis my-aiven-project google-europe-west1 startup-4 RUNNING \nTo view the details of the Secret, use the following command:
kubectl describe secret redis-token\nYou can use the jq to quickly decode the Secret:
kubectl get secret redis-token -o json | jq '.data | map_values(@base64d)'\nThe output is similar to the following:
{\n \"REDIS_HOST\": \"<secret>\",\n \"REDIS_PORT\": \"<secret>\",\n \"REDIS_USER\": \"<secret>\",\n \"REDIS_PASSWORD\": \"<secret>\",\n}\nRedis is the Schema for the redis API.
Exposes secret keys
REDIS_HOST, REDIS_PORT, REDIS_USER, REDIS_PASSWORD.
Required
apiVersion (string). Value aiven.io/v1alpha1.kind (string). Value Redis.metadata (object). Data that identifies the object, including a name string and optional namespace.spec (object). RedisSpec defines the desired state of Redis. See below for nested schema.Appears on Redis.
RedisSpec defines the desired state of Redis.
Required
plan (string, MaxLength: 128). Subscription plan.project (string, Immutable, Pattern: ^[a-zA-Z0-9_-]+$, MaxLength: 63). Identifies the project this resource belongs to.Optional
authSecretRef (object). Authentication reference to Aiven token in a secret. See below for nested schema.cloudName (string, MaxLength: 256). Cloud the service runs in.connInfoSecretTarget (object). Secret configuration. See below for nested schema.connInfoSecretTargetDisabled (boolean, Immutable). When true, the secret containing connection information will not be created, defaults to false. This field cannot be changed after resource creation.disk_space (string, Pattern: (?i)^[1-9][0-9]*(GiB|G)?$). The disk space of the service, possible values depend on the service type, the cloud provider and the project. Reducing will result in the service re-balancing. The removal of this field does not change the value.maintenanceWindowDow (string, Enum: monday, tuesday, wednesday, thursday, friday, saturday, sunday). Day of week when maintenance operations should be performed. One monday, tuesday, wednesday, etc.maintenanceWindowTime (string, MaxLength: 8). Time of day when maintenance operations should be performed. UTC time in HH:mm:ss format.projectVPCRef (object). ProjectVPCRef reference to ProjectVPC resource to use its ID as ProjectVPCID automatically. See below for nested schema.projectVpcId (string, MaxLength: 36). Identifier of the VPC the service should be in, if any.serviceIntegrations (array of objects, Immutable, MaxItems: 1). Service integrations to specify when creating a service. Not applied after initial service creation. See below for nested schema.tags (object, AdditionalProperties: string). Tags are key-value pairs that allow you to categorize services.technicalEmails (array of objects, MaxItems: 10). Defines the email addresses that will receive alerts about upcoming maintenance updates or warnings about service instability. See below for nested schema.terminationProtection (boolean). Prevent service from being deleted. It is recommended to have this enabled for all services.userConfig (object). Redis specific user configuration options. See below for nested schema.Appears on spec.
Authentication reference to Aiven token in a secret.
Required
key (string, MinLength: 1).name (string, MinLength: 1).Appears on spec.
Secret configuration.
Required
name (string, Immutable). Name of the secret resource to be created. By default, it is equal to the resource name.Optional
annotations (object, AdditionalProperties: string). Annotations added to the secret.labels (object, AdditionalProperties: string). Labels added to the secret.prefix (string). Prefix for the secret's keys. Added \"as is\" without any transformations. By default, is equal to the kind name in uppercase + underscore, e.g. KAFKA_, REDIS_, etc.Appears on spec.
ProjectVPCRef reference to ProjectVPC resource to use its ID as ProjectVPCID automatically.
Required
name (string, MinLength: 1).Optional
namespace (string, MinLength: 1).Appears on spec.
Service integrations to specify when creating a service. Not applied after initial service creation.
Required
integrationType (string, Enum: read_replica).sourceServiceName (string, MinLength: 1, MaxLength: 64).Appears on spec.
Defines the email addresses that will receive alerts about upcoming maintenance updates or warnings about service instability.
Required
email (string). Email address.Appears on spec.
Redis specific user configuration options.
Optional
additional_backup_regions (array of strings, MaxItems: 1). Additional Cloud Regions for Backup Replication.ip_filter (array of objects, MaxItems: 1024). Allow incoming connections from CIDR address block, e.g. 10.20.0.0/16. See below for nested schema.migration (object). Migrate data from existing server. See below for nested schema.private_access (object). Allow access to selected service ports from private networks. See below for nested schema.privatelink_access (object). Allow access to selected service components through Privatelink. See below for nested schema.project_to_fork_from (string, Immutable, Pattern: ^[a-z][-a-z0-9]{0,63}$|^$, MaxLength: 63). Name of another project to fork a service from. This has effect only when a new service is being created.public_access (object). Allow access to selected service ports from the public Internet. See below for nested schema.recovery_basebackup_name (string, Pattern: ^[a-zA-Z0-9-_:.]+$, MaxLength: 128). Name of the basebackup to restore in forked service.redis_acl_channels_default (string, Enum: allchannels, resetchannels). Determines default pub/sub channels' ACL for new users if ACL is not supplied. When this option is not defined, all_channels is assumed to keep backward compatibility. This option doesn't affect Redis configuration acl-pubsub-default.redis_io_threads (integer, Minimum: 1, Maximum: 32). Set Redis IO thread count. Changing this will cause a restart of the Redis service.redis_lfu_decay_time (integer, Minimum: 1, Maximum: 120). LFU maxmemory-policy counter decay time in minutes.redis_lfu_log_factor (integer, Minimum: 0, Maximum: 100). Counter logarithm factor for volatile-lfu and allkeys-lfu maxmemory-policies.redis_maxmemory_policy (string, Enum: noeviction, allkeys-lru, volatile-lru, allkeys-random, volatile-random, volatile-ttl, volatile-lfu, allkeys-lfu). Redis maxmemory-policy.redis_notify_keyspace_events (string, Pattern: ^[KEg\\$lshzxentdmA]*$, MaxLength: 32). Set notify-keyspace-events option.redis_number_of_databases (integer, Minimum: 1, Maximum: 128). Set number of Redis databases. Changing this will cause a restart of the Redis service.redis_persistence (string, Enum: off, rdb). When persistence is rdb, Redis does RDB dumps each 10 minutes if any key is changed. Also RDB dumps are done according to the backup schedule for backup purposes. When persistence is off, no RDB dumps or backups are done, so data can be lost at any moment if the service is restarted for any reason, or if the service is powered off. Also, the service can't be forked.redis_pubsub_client_output_buffer_limit (integer, Minimum: 32, Maximum: 512). Set output buffer limit for pub / sub clients in MB. The value is the hard limit, the soft limit is 1/4 of the hard limit. When setting the limit, be mindful of the available memory in the selected service plan.redis_ssl (boolean). Require SSL to access Redis.redis_timeout (integer, Minimum: 0, Maximum: 31536000). Redis idle connection timeout in seconds.redis_version (string, Enum: 7.0). Redis major version.service_log (boolean). Store logs for the service so that they are available in the HTTP API and console.service_to_fork_from (string, Immutable, Pattern: ^[a-z][-a-z0-9]{0,63}$|^$, MaxLength: 64). Name of another service to fork from. This has effect only when a new service is being created.static_ips (boolean). Use static public IP addresses.Appears on spec.userConfig.
Allow incoming connections from CIDR address block, e.g. 10.20.0.0/16.
Required
network (string, MaxLength: 43). CIDR address block.Optional
description (string, MaxLength: 1024). Description for IP filter list entry.Appears on spec.userConfig.
Migrate data from existing server.
Required
host (string, MaxLength: 255). Hostname or IP address of the server where to migrate data from.port (integer, Minimum: 1, Maximum: 65535). Port number of the server where to migrate data from.Optional
dbname (string, MaxLength: 63). Database name for bootstrapping the initial connection.ignore_dbs (string, MaxLength: 2048). Comma-separated list of databases, which should be ignored during migration (supported by MySQL and PostgreSQL only at the moment).method (string, Enum: dump, replication). The migration method to be used (currently supported only by Redis, Dragonfly, MySQL and PostgreSQL service types).password (string, MaxLength: 256). Password for authentication with the server where to migrate data from.ssl (boolean). The server where to migrate data from is secured with SSL.username (string, MaxLength: 256). User name for authentication with the server where to migrate data from.Appears on spec.userConfig.
Allow access to selected service ports from private networks.
Optional
prometheus (boolean). Allow clients to connect to prometheus with a DNS name that always resolves to the service's private IP addresses. Only available in certain network locations.redis (boolean). Allow clients to connect to redis with a DNS name that always resolves to the service's private IP addresses. Only available in certain network locations.Appears on spec.userConfig.
Allow access to selected service components through Privatelink.
Optional
prometheus (boolean). Enable prometheus.redis (boolean). Enable redis.Appears on spec.userConfig.
Allow access to selected service ports from the public Internet.
Optional
prometheus (boolean). Allow clients to connect to prometheus from the public internet for service nodes that are in a project VPC or another type of private network.redis (boolean). Allow clients to connect to redis from the public internet for service nodes that are in a project VPC or another type of private network.apiVersion: aiven.io/v1alpha1\nkind: ServiceIntegration\nmetadata:\n name: my-service-integration\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n project: aiven-project-name\n integrationType: clickhouse_postgresql\n sourceServiceName: my-pg\n destinationServiceName: my-clickhouse\n\n clickhousePostgresql:\n databases:\n - database: defaultdb\n schema: public\n\n---\n\napiVersion: aiven.io/v1alpha1\nkind: Clickhouse\nmetadata:\n name: my-clickhouse\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n project: aiven-project-name\n cloudName: google-europe-west1\n plan: startup-16\n maintenanceWindowDow: friday\n maintenanceWindowTime: 23:00:00\n\n---\n\napiVersion: aiven.io/v1alpha1\nkind: PostgreSQL\nmetadata:\n name: my-pg\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n project: aiven-project-name\n cloudName: google-europe-west1\n plan: startup-4\n maintenanceWindowDow: friday\n maintenanceWindowTime: 23:00:00\napiVersion: aiven.io/v1alpha1\nkind: ServiceIntegration\nmetadata:\n name: my-service-integration\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n project: aiven-project-name\n integrationType: datadog\n sourceServiceName: my-pg\n destinationEndpointId: destination-endpoint-id\n\n datadog:\n datadog_dbm_enabled: True\n datadog_tags:\n - tag: env\n comment: test\n\n---\n\napiVersion: aiven.io/v1alpha1\nkind: PostgreSQL\nmetadata:\n name: my-pg\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n project: aiven-project-name\n cloudName: google-europe-west1\n plan: startup-4\napiVersion: aiven.io/v1alpha1\nkind: ServiceIntegration\nmetadata:\n name: my-service-integration\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n project: aiven-project-name\n integrationType: kafka_connect\n sourceServiceName: my-kafka\n destinationServiceName: my-kafka-connect\n\n kafkaConnect:\n kafka_connect:\n group_id: connect\n status_storage_topic: __connect_status\n offset_storage_topic: __connect_offsets\n\n---\n\napiVersion: aiven.io/v1alpha1\nkind: Kafka\nmetadata:\n name: my-kafka\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n project: aiven-project-name\n cloudName: google-europe-west1\n plan: business-4\n\n---\n\napiVersion: aiven.io/v1alpha1\nkind: KafkaConnect\nmetadata:\n name: my-kafka-connect\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n project: aiven-project-name\n cloudName: google-europe-west1\n plan: business-4\n\n userConfig:\n kafka_connect:\n consumer_isolation_level: read_committed\n public_access:\n kafka_connect: true\napiVersion: aiven.io/v1alpha1\nkind: ServiceIntegration\nmetadata:\n name: my-service-integration\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n project: aiven-project-name\n integrationType: kafka_logs\n sourceServiceName: my-kafka\n destinationServiceName: my-kafka\n\n kafkaLogs:\n kafka_topic: my-kafka-topic\n\n---\n\napiVersion: aiven.io/v1alpha1\nkind: Kafka\nmetadata:\n name: my-kafka\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n project: aiven-project-name\n cloudName: google-europe-west1\n plan: business-4\n\n---\n\napiVersion: aiven.io/v1alpha1\nkind: KafkaTopic\nmetadata:\n name: my-kafka-topic\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n project: aiven-project-name\n serviceName: my-kafka\n replication: 2\n partitions: 1\nInfo
To create this resource, a Secret containing Aiven token must be created first.
Apply the resource with:
kubectl apply -f example.yaml\nVerify the newly created ServiceIntegration:
kubectl get serviceintegrations my-service-integration\nThe output is similar to the following:
Name Project Type Source Service Name Destination Service Name \nmy-service-integration aiven-project-name clickhouse_postgresql my-pg my-clickhouse \nServiceIntegration is the Schema for the serviceintegrations API.
Required
apiVersion (string). Value aiven.io/v1alpha1.kind (string). Value ServiceIntegration.metadata (object). Data that identifies the object, including a name string and optional namespace.spec (object). ServiceIntegrationSpec defines the desired state of ServiceIntegration. See below for nested schema.Appears on ServiceIntegration.
ServiceIntegrationSpec defines the desired state of ServiceIntegration.
Required
integrationType (string, Enum: alertmanager, autoscaler, caching, cassandra_cross_service_cluster, clickhouse_kafka, clickhouse_postgresql, dashboard, datadog, datasource, external_aws_cloudwatch_logs, external_aws_cloudwatch_metrics, external_elasticsearch_logs, external_google_cloud_logging, external_opensearch_logs, flink, flink_external_kafka, flink_external_postgresql, internal_connectivity, jolokia, kafka_connect, kafka_logs, kafka_mirrormaker, logs, m3aggregator, m3coordinator, metrics, opensearch_cross_cluster_replication, opensearch_cross_cluster_search, prometheus, read_replica, rsyslog, schema_registry_proxy, stresstester, thanosquery, thanosstore, vmalert, Immutable). Type of the service integration accepted by Aiven API. Some values may not be supported by the operator.project (string, Immutable, Pattern: ^[a-zA-Z0-9_-]+$, MaxLength: 63). Identifies the project this resource belongs to.Optional
authSecretRef (object). Authentication reference to Aiven token in a secret. See below for nested schema.clickhouseKafka (object). Clickhouse Kafka configuration values. See below for nested schema.clickhousePostgresql (object). Clickhouse PostgreSQL configuration values. See below for nested schema.datadog (object). Datadog specific user configuration options. See below for nested schema.destinationEndpointId (string, Immutable, MaxLength: 36). Destination endpoint for the integration (if any).destinationProjectName (string, Immutable, MaxLength: 63). Destination project for the integration (if any).destinationServiceName (string, Immutable, MaxLength: 64). Destination service for the integration (if any).externalAWSCloudwatchMetrics (object). External AWS CloudWatch Metrics integration Logs configuration values. See below for nested schema.kafkaConnect (object). Kafka Connect service configuration values. See below for nested schema.kafkaLogs (object). Kafka logs configuration values. See below for nested schema.kafkaMirrormaker (object). Kafka MirrorMaker configuration values. See below for nested schema.logs (object). Logs configuration values. See below for nested schema.metrics (object). Metrics configuration values. See below for nested schema.sourceEndpointID (string, Immutable, MaxLength: 36). Source endpoint for the integration (if any).sourceProjectName (string, Immutable, MaxLength: 63). Source project for the integration (if any).sourceServiceName (string, Immutable, MaxLength: 64). Source service for the integration (if any).Appears on spec.
Authentication reference to Aiven token in a secret.
Required
key (string, MinLength: 1).name (string, MinLength: 1).Appears on spec.
Clickhouse Kafka configuration values.
Required
tables (array of objects, MaxItems: 100). Tables to create. See below for nested schema.Appears on spec.clickhouseKafka.
Tables to create.
Required
columns (array of objects, MaxItems: 100). Table columns. See below for nested schema.data_format (string, Enum: Avro, CSV, JSONAsString, JSONCompactEachRow, JSONCompactStringsEachRow, JSONEachRow, JSONStringsEachRow, MsgPack, TSKV, TSV, TabSeparated, RawBLOB, AvroConfluent, Parquet). Message data format.group_name (string, MinLength: 1, MaxLength: 249). Kafka consumers group.name (string, MinLength: 1, MaxLength: 40). Name of the table.topics (array of objects, MaxItems: 100). Kafka topics. See below for nested schema.Optional
auto_offset_reset (string, Enum: smallest, earliest, beginning, largest, latest, end). Action to take when there is no initial offset in offset store or the desired offset is out of range.date_time_input_format (string, Enum: basic, best_effort, best_effort_us). Method to read DateTime from text input formats.handle_error_mode (string, Enum: default, stream). How to handle errors for Kafka engine.max_block_size (integer, Minimum: 0, Maximum: 1000000000). Number of row collected by poll(s) for flushing data from Kafka.max_rows_per_message (integer, Minimum: 1, Maximum: 1000000000). The maximum number of rows produced in one kafka message for row-based formats.num_consumers (integer, Minimum: 1, Maximum: 10). The number of consumers per table per replica.poll_max_batch_size (integer, Minimum: 0, Maximum: 1000000000). Maximum amount of messages to be polled in a single Kafka poll.skip_broken_messages (integer, Minimum: 0, Maximum: 1000000000). Skip at least this number of broken messages from Kafka topic per block.Appears on spec.clickhouseKafka.tables.
Table columns.
Required
name (string, MinLength: 1, MaxLength: 40). Column name.type (string, MinLength: 1, MaxLength: 1000). Column type.Appears on spec.clickhouseKafka.tables.
Kafka topics.
Required
name (string, MinLength: 1, MaxLength: 249). Name of the topic.Appears on spec.
Clickhouse PostgreSQL configuration values.
Required
databases (array of objects, MaxItems: 10). Databases to expose. See below for nested schema.Appears on spec.clickhousePostgresql.
Databases to expose.
Optional
database (string, MinLength: 1, MaxLength: 63). PostgreSQL database to expose.schema (string, MinLength: 1, MaxLength: 63). PostgreSQL schema to expose.Appears on spec.
Datadog specific user configuration options.
Optional
datadog_dbm_enabled (boolean). Enable Datadog Database Monitoring.datadog_pgbouncer_enabled (boolean). Enable Datadog PgBouncer Metric Tracking.datadog_tags (array of objects, MaxItems: 32). Custom tags provided by user. See below for nested schema.exclude_consumer_groups (array of strings, MaxItems: 1024). List of custom metrics.exclude_topics (array of strings, MaxItems: 1024). List of topics to exclude.include_consumer_groups (array of strings, MaxItems: 1024). List of custom metrics.include_topics (array of strings, MaxItems: 1024). List of topics to include.kafka_custom_metrics (array of strings, MaxItems: 1024). List of custom metrics.max_jmx_metrics (integer, Minimum: 10, Maximum: 100000). Maximum number of JMX metrics to send.mirrormaker_custom_metrics (array of strings, MaxItems: 1024). List of custom metrics.opensearch (object). Datadog Opensearch Options. See below for nested schema.redis (object). Datadog Redis Options. See below for nested schema.Appears on spec.datadog.
Custom tags provided by user.
Required
tag (string, MinLength: 1, MaxLength: 200). Tag format and usage are described here: https://docs.datadoghq.com/getting_started/tagging. Tags with prefix aiven- are reserved for Aiven.Optional
comment (string, MaxLength: 1024). Optional tag explanation.Appears on spec.datadog.
Datadog Opensearch Options.
Optional
cluster_stats_enabled (boolean). Enable Datadog Opensearch Cluster Monitoring.index_stats_enabled (boolean). Enable Datadog Opensearch Index Monitoring.pending_task_stats_enabled (boolean). Enable Datadog Opensearch Pending Task Monitoring.pshard_stats_enabled (boolean). Enable Datadog Opensearch Primary Shard Monitoring.Appears on spec.datadog.
Datadog Redis Options.
Required
command_stats_enabled (boolean). Enable command_stats option in the agent's configuration.Appears on spec.
External AWS CloudWatch Metrics integration Logs configuration values.
Optional
dropped_metrics (array of objects, MaxItems: 1024). Metrics to not send to AWS CloudWatch (takes precedence over extra_metrics). See below for nested schema.extra_metrics (array of objects, MaxItems: 1024). Metrics to allow through to AWS CloudWatch (in addition to default metrics). See below for nested schema.Appears on spec.externalAWSCloudwatchMetrics.
Metrics to not send to AWS CloudWatch (takes precedence over extra_metrics).
Required
field (string, MaxLength: 1000). Identifier of a value in the metric.metric (string, MaxLength: 1000). Identifier of the metric.Appears on spec.externalAWSCloudwatchMetrics.
Metrics to allow through to AWS CloudWatch (in addition to default metrics).
Required
field (string, MaxLength: 1000). Identifier of a value in the metric.metric (string, MaxLength: 1000). Identifier of the metric.Appears on spec.
Kafka Connect service configuration values.
Required
kafka_connect (object). Kafka Connect service configuration values. See below for nested schema.Appears on spec.kafkaConnect.
Kafka Connect service configuration values.
Optional
config_storage_topic (string, MaxLength: 249). The name of the topic where connector and task configuration data are stored.This must be the same for all workers with the same group_id.group_id (string, MaxLength: 249). A unique string that identifies the Connect cluster group this worker belongs to.offset_storage_topic (string, MaxLength: 249). The name of the topic where connector and task configuration offsets are stored.This must be the same for all workers with the same group_id.status_storage_topic (string, MaxLength: 249). The name of the topic where connector and task configuration status updates are stored.This must be the same for all workers with the same group_id.Appears on spec.
Kafka logs configuration values.
Required
kafka_topic (string, MinLength: 1, MaxLength: 249). Topic name.Optional
selected_log_fields (array of strings, MaxItems: 5). The list of logging fields that will be sent to the integration logging service. The MESSAGE and timestamp fields are always sent.Appears on spec.
Kafka MirrorMaker configuration values.
Optional
cluster_alias (string, Pattern: ^[a-zA-Z0-9_.-]+$, MaxLength: 128). The alias under which the Kafka cluster is known to MirrorMaker. Can contain the following symbols: ASCII alphanumerics, ., _, and -.kafka_mirrormaker (object). Kafka MirrorMaker configuration values. See below for nested schema.Appears on spec.kafkaMirrormaker.
Kafka MirrorMaker configuration values.
Optional
consumer_auto_offset_reset (string, Enum: earliest, latest). Set where consumer starts to consume data. Value earliest: Start replication from the earliest offset. Value latest: Start replication from the latest offset. Default is earliest.consumer_fetch_min_bytes (integer, Minimum: 1, Maximum: 5242880). The minimum amount of data the server should return for a fetch request.consumer_max_poll_records (integer, Minimum: 100, Maximum: 20000). Set consumer max.poll.records. The default is 500.producer_batch_size (integer, Minimum: 0, Maximum: 5242880). The batch size in bytes producer will attempt to collect before publishing to broker.producer_buffer_memory (integer, Minimum: 5242880, Maximum: 134217728). The amount of bytes producer can use for buffering data before publishing to broker.producer_compression_type (string, Enum: gzip, snappy, lz4, zstd, none). Specify the default compression type for producers. This configuration accepts the standard compression codecs (gzip, snappy, lz4, zstd). It additionally accepts none which is the default and equivalent to no compression.producer_linger_ms (integer, Minimum: 0, Maximum: 5000). The linger time (ms) for waiting new data to arrive for publishing.producer_max_request_size (integer, Minimum: 0, Maximum: 268435456). The maximum request size in bytes.Appears on spec.
Logs configuration values.
Optional
elasticsearch_index_days_max (integer, Minimum: 1, Maximum: 10000). Elasticsearch index retention limit.elasticsearch_index_prefix (string, MinLength: 1, MaxLength: 1024). Elasticsearch index prefix.selected_log_fields (array of strings, MaxItems: 5). The list of logging fields that will be sent to the integration logging service. The MESSAGE and timestamp fields are always sent.Appears on spec.
Metrics configuration values.
Optional
database (string, Pattern: ^[_A-Za-z0-9][-_A-Za-z0-9]{0,39}$, MaxLength: 40). Name of the database where to store metric datapoints. Only affects PostgreSQL destinations. Defaults to metrics. Note that this must be the same for all metrics integrations that write data to the same PostgreSQL service.retention_days (integer, Minimum: 0, Maximum: 10000). Number of days to keep old metrics. Only affects PostgreSQL destinations. Set to 0 for no automatic cleanup. Defaults to 30 days.ro_username (string, Pattern: ^[_A-Za-z0-9][-._A-Za-z0-9]{0,39}$, MaxLength: 40). Name of a user that can be used to read metrics. This will be used for Grafana integration (if enabled) to prevent Grafana users from making undesired changes. Only affects PostgreSQL destinations. Defaults to metrics_reader. Note that this must be the same for all metrics integrations that write data to the same PostgreSQL service.source_mysql (object). Configuration options for metrics where source service is MySQL. See below for nested schema.username (string, Pattern: ^[_A-Za-z0-9][-._A-Za-z0-9]{0,39}$, MaxLength: 40). Name of the user used to write metrics. Only affects PostgreSQL destinations. Defaults to metrics_writer. Note that this must be the same for all metrics integrations that write data to the same PostgreSQL service.Appears on spec.metrics.
Configuration options for metrics where source service is MySQL.
Required
telegraf (object). Configuration options for Telegraf MySQL input plugin. See below for nested schema.Appears on spec.metrics.source_mysql.
Configuration options for Telegraf MySQL input plugin.
Optional
gather_event_waits (boolean). Gather metrics from PERFORMANCE_SCHEMA.EVENT_WAITS.gather_file_events_stats (boolean). gather metrics from PERFORMANCE_SCHEMA.FILE_SUMMARY_BY_EVENT_NAME.gather_index_io_waits (boolean). Gather metrics from PERFORMANCE_SCHEMA.TABLE_IO_WAITS_SUMMARY_BY_INDEX_USAGE.gather_info_schema_auto_inc (boolean). Gather auto_increment columns and max values from information schema.gather_innodb_metrics (boolean). Gather metrics from INFORMATION_SCHEMA.INNODB_METRICS.gather_perf_events_statements (boolean). Gather metrics from PERFORMANCE_SCHEMA.EVENTS_STATEMENTS_SUMMARY_BY_DIGEST.gather_process_list (boolean). Gather thread state counts from INFORMATION_SCHEMA.PROCESSLIST.gather_slave_status (boolean). Gather metrics from SHOW SLAVE STATUS command output.gather_table_io_waits (boolean). Gather metrics from PERFORMANCE_SCHEMA.TABLE_IO_WAITS_SUMMARY_BY_TABLE.gather_table_lock_waits (boolean). Gather metrics from PERFORMANCE_SCHEMA.TABLE_LOCK_WAITS.gather_table_schema (boolean). Gather metrics from INFORMATION_SCHEMA.TABLES.perf_events_statements_digest_text_limit (integer, Minimum: 1, Maximum: 2048). Truncates digest text from perf_events_statements into this many characters.perf_events_statements_limit (integer, Minimum: 1, Maximum: 4000). Limits metrics from perf_events_statements.perf_events_statements_time_limit (integer, Minimum: 1, Maximum: 2592000). Only include perf_events_statements whose last seen is less than this many seconds.apiVersion: aiven.io/v1alpha1\nkind: ServiceIntegrationEndpoint\nmetadata:\n name: my-service-integration-endpoint\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n project: aiven-project-name\n endpointName: my-external-postgresql\n endpointType: external_postgresql\n\n externalPostgresql:\n username: username\n password: password\n host: example.example\n port: 5432\n ssl_mode: require\napiVersion: aiven.io/v1alpha1\nkind: ServiceIntegrationEndpoint\nmetadata:\n name: my-service-integration-endpoint\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n project: aiven-project-name\n endpointName: my-external-schema-registry\n endpointType: external_schema_registry\n\n externalSchemaRegistry:\n url: https://schema-registry.example.com:8081\n authentication: basic\n basic_auth_username: username\n basic_auth_password: password\nInfo
To create this resource, a Secret containing Aiven token must be created first.
Apply the resource with:
kubectl apply -f example.yaml\nVerify the newly created ServiceIntegrationEndpoint:
kubectl get serviceintegrationendpoints my-service-integration-endpoint\nThe output is similar to the following:
Name Project Endpoint Name Endpoint Type ID \nmy-service-integration-endpoint aiven-project-name my-external-postgresql external_postgresql <id> \nServiceIntegrationEndpoint is the Schema for the serviceintegrationendpoints API.
Required
apiVersion (string). Value aiven.io/v1alpha1.kind (string). Value ServiceIntegrationEndpoint.metadata (object). Data that identifies the object, including a name string and optional namespace.spec (object). ServiceIntegrationEndpointSpec defines the desired state of ServiceIntegrationEndpoint. See below for nested schema.Appears on ServiceIntegrationEndpoint.
ServiceIntegrationEndpointSpec defines the desired state of ServiceIntegrationEndpoint.
Required
endpointType (string, Enum: autoscaler, datadog, external_aws_cloudwatch_logs, external_aws_cloudwatch_metrics, external_aws_s3, external_clickhouse, external_elasticsearch_logs, external_google_cloud_bigquery, external_google_cloud_logging, external_kafka, external_mysql, external_opensearch_logs, external_postgresql, external_redis, external_schema_registry, external_sumologic_logs, jolokia, prometheus, rsyslog, Immutable). Type of the service integration endpoint.project (string, Immutable, Pattern: ^[a-zA-Z0-9_-]+$, MaxLength: 63). Identifies the project this resource belongs to.Optional
authSecretRef (object). Authentication reference to Aiven token in a secret. See below for nested schema.datadog (object). Datadog configuration values. See below for nested schema.endpointName (string, Immutable, MaxLength: 36). Source endpoint for the integration (if any).externalAWSCloudwatchLogs (object). ExternalAwsCloudwatchLogs configuration values. See below for nested schema.externalAWSCloudwatchMetrics (object). ExternalAwsCloudwatchMetrics configuration values. See below for nested schema.externalElasticsearchLogs (object). ExternalElasticsearchLogs configuration values. See below for nested schema.externalGoogleCloudBigquery (object). ExternalGoogleCloudBigquery configuration values. See below for nested schema.externalGoogleCloudLogging (object). ExternalGoogleCloudLogging configuration values. See below for nested schema.externalKafka (object). ExternalKafka configuration values. See below for nested schema.externalOpensearchLogs (object). ExternalOpensearchLogs configuration values. See below for nested schema.externalPostgresql (object). ExternalPostgresql configuration values. See below for nested schema.externalSchemaRegistry (object). ExternalSchemaRegistry configuration values. See below for nested schema.jolokia (object). Jolokia configuration values. See below for nested schema.prometheus (object). Prometheus configuration values. See below for nested schema.rsyslog (object). Rsyslog configuration values. See below for nested schema.Appears on spec.
Authentication reference to Aiven token in a secret.
Required
key (string, MinLength: 1).name (string, MinLength: 1).Appears on spec.
Datadog configuration values.
Required
datadog_api_key (string, Pattern: ^[A-Za-z0-9]{1,256}$, MinLength: 1, MaxLength: 256). Datadog API key.Optional
datadog_tags (array of objects, MaxItems: 32). Custom tags provided by user. See below for nested schema.disable_consumer_stats (boolean). Disable consumer group metrics.kafka_consumer_check_instances (integer, Minimum: 1, Maximum: 100). Number of separate instances to fetch kafka consumer statistics with.kafka_consumer_stats_timeout (integer, Minimum: 2, Maximum: 300). Number of seconds that datadog will wait to get consumer statistics from brokers.max_partition_contexts (integer, Minimum: 200, Maximum: 200000). Maximum number of partition contexts to send.site (string, Enum: datadoghq.com, datadoghq.eu, us3.datadoghq.com, us5.datadoghq.com, ddog-gov.com, ap1.datadoghq.com). Datadog intake site. Defaults to datadoghq.com.Appears on spec.datadog.
Custom tags provided by user.
Required
tag (string, MinLength: 1, MaxLength: 200). Tag format and usage are described here: https://docs.datadoghq.com/getting_started/tagging. Tags with prefix aiven- are reserved for Aiven.Optional
comment (string, MaxLength: 1024). Optional tag explanation.Appears on spec.
ExternalAwsCloudwatchLogs configuration values.
Required
access_key (string, MaxLength: 4096). AWS access key. Required permissions are logs:CreateLogGroup, logs:CreateLogStream, logs:PutLogEvents and logs:DescribeLogStreams.region (string, MaxLength: 32). AWS region.secret_key (string, MaxLength: 4096). AWS secret key.Optional
log_group_name (string, Pattern: ^[\\.\\-_/#A-Za-z0-9]+$, MinLength: 1, MaxLength: 512). AWS CloudWatch log group name.Appears on spec.
ExternalAwsCloudwatchMetrics configuration values.
Required
access_key (string, MaxLength: 4096). AWS access key. Required permissions are cloudwatch:PutMetricData.namespace (string, MinLength: 1, MaxLength: 255). AWS CloudWatch Metrics Namespace.region (string, MaxLength: 32). AWS region.secret_key (string, MaxLength: 4096). AWS secret key.Appears on spec.
ExternalElasticsearchLogs configuration values.
Required
index_prefix (string, Pattern: ^[a-z0-9][a-z0-9-_.]+$, MinLength: 1, MaxLength: 1000). Elasticsearch index prefix.url (string, MinLength: 12, MaxLength: 2048). Elasticsearch connection URL.Optional
ca (string, MaxLength: 16384). PEM encoded CA certificate.index_days_max (integer, Minimum: 1, Maximum: 10000). Maximum number of days of logs to keep.timeout (number, Minimum: 10, Maximum: 120). Elasticsearch request timeout limit.Appears on spec.
ExternalGoogleCloudBigquery configuration values.
Required
project_id (string, MinLength: 6, MaxLength: 30). GCP project id.service_account_credentials (string, MaxLength: 4096). This is a JSON object with the fields documented in https://cloud.google.com/iam/docs/creating-managing-service-account-keys .Appears on spec.
ExternalGoogleCloudLogging configuration values.
Required
log_id (string, MaxLength: 512). Google Cloud Logging log id.project_id (string, MinLength: 6, MaxLength: 30). GCP project id.service_account_credentials (string, MaxLength: 4096). This is a JSON object with the fields documented in https://cloud.google.com/iam/docs/creating-managing-service-account-keys .Appears on spec.
ExternalKafka configuration values.
Required
bootstrap_servers (string, MinLength: 3, MaxLength: 256). Bootstrap servers.security_protocol (string, Enum: PLAINTEXT, SSL, SASL_PLAINTEXT, SASL_SSL). Security protocol.Optional
sasl_mechanism (string, Enum: PLAIN, SCRAM-SHA-256, SCRAM-SHA-512). SASL mechanism used for connections to the Kafka server.sasl_plain_password (string, MinLength: 1, MaxLength: 256). Password for SASL PLAIN mechanism in the Kafka server.sasl_plain_username (string, MinLength: 1, MaxLength: 256). Username for SASL PLAIN mechanism in the Kafka server.ssl_ca_cert (string, MaxLength: 16384). PEM-encoded CA certificate.ssl_client_cert (string, MaxLength: 16384). PEM-encoded client certificate.ssl_client_key (string, MaxLength: 16384). PEM-encoded client key.ssl_endpoint_identification_algorithm (string, Enum: https). The endpoint identification algorithm to validate server hostname using server certificate.Appears on spec.
ExternalOpensearchLogs configuration values.
Required
index_prefix (string, Pattern: ^[a-z0-9][a-z0-9-_.]+$, MinLength: 1, MaxLength: 1000). OpenSearch index prefix.url (string, MinLength: 12, MaxLength: 2048). OpenSearch connection URL.Optional
ca (string, MaxLength: 16384). PEM encoded CA certificate.index_days_max (integer, Minimum: 1, Maximum: 10000). Maximum number of days of logs to keep.timeout (number, Minimum: 10, Maximum: 120). OpenSearch request timeout limit.Appears on spec.
ExternalPostgresql configuration values.
Required
host (string, MaxLength: 255). Hostname or IP address of the server.port (integer, Minimum: 1, Maximum: 65535). Port number of the server.username (string, MaxLength: 256). User name.Optional
default_database (string, Pattern: ^[_A-Za-z0-9][-_A-Za-z0-9]{0,62}$, MaxLength: 63). Default database.password (string, MaxLength: 256). Password.ssl_client_certificate (string, MaxLength: 16384). Client certificate.ssl_client_key (string, MaxLength: 16384). Client key.ssl_mode (string, Enum: require, verify-ca, verify-full). SSL mode to use for the connection. Please note that Aiven requires TLS for all connections to external PostgreSQL services.ssl_root_cert (string, MaxLength: 16384). SSL Root Cert.Appears on spec.
ExternalSchemaRegistry configuration values.
Required
authentication (string, Enum: none, basic). Authentication method.url (string, MaxLength: 2048). Schema Registry URL.Optional
basic_auth_password (string, MaxLength: 256). Basic authentication password.basic_auth_username (string, MaxLength: 256). Basic authentication user name.Appears on spec.
Jolokia configuration values.
Optional
basic_auth_password (string, MinLength: 8, MaxLength: 64). Jolokia basic authentication password.basic_auth_username (string, Pattern: ^[a-z0-9\\-@_]{5,32}$, MinLength: 5, MaxLength: 32). Jolokia basic authentication username.Appears on spec.
Prometheus configuration values.
Optional
basic_auth_password (string, MinLength: 8, MaxLength: 64). Prometheus basic authentication password.basic_auth_username (string, Pattern: ^[a-z0-9\\-@_]{5,32}$, MinLength: 5, MaxLength: 32). Prometheus basic authentication username.Appears on spec.
Rsyslog configuration values.
Required
format (string, Enum: rfc5424, rfc3164, custom). Message format.port (integer, Minimum: 1, Maximum: 65535). Rsyslog server port.server (string, MinLength: 4, MaxLength: 255). Rsyslog server IP address or hostname.tls (boolean). Require TLS.Optional
ca (string, MaxLength: 16384). PEM encoded CA certificate.cert (string, MaxLength: 16384). PEM encoded client certificate.key (string, MaxLength: 16384). PEM encoded client key.logline (string, Pattern: ^[ -~\\t]+$, MinLength: 1, MaxLength: 512). Custom syslog message format.max_message_size (integer, Minimum: 2048, Maximum: 2147483647). Rsyslog max message size.sd (string, MaxLength: 1024). Structured data block for log message.apiVersion: aiven.io/v1alpha1\nkind: ServiceUser\nmetadata:\n name: my-service-user\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n connInfoSecretTarget:\n name: service-user-secret\n prefix: MY_SECRET_PREFIX_\n annotations:\n foo: bar\n labels:\n baz: egg\n\n project: aiven-project-name\n serviceName: my-service-name\nInfo
To create this resource, a Secret containing Aiven token must be created first.
Apply the resource with:
kubectl apply -f example.yaml\nVerify the newly created ServiceUser:
kubectl get serviceusers my-service-user\nThe output is similar to the following:
Name Service Name Project \nmy-service-user my-service-name aiven-project-name \nTo view the details of the Secret, use the following command:
kubectl describe secret service-user-secret\nYou can use the jq to quickly decode the Secret:
kubectl get secret service-user-secret -o json | jq '.data | map_values(@base64d)'\nThe output is similar to the following:
{\n \"SERVICEUSER_HOST\": \"<secret>\",\n \"SERVICEUSER_PORT\": \"<secret>\",\n \"SERVICEUSER_USERNAME\": \"<secret>\",\n \"SERVICEUSER_PASSWORD\": \"<secret>\",\n \"SERVICEUSER_CA_CERT\": \"<secret>\",\n \"SERVICEUSER_ACCESS_CERT\": \"<secret>\",\n \"SERVICEUSER_ACCESS_KEY\": \"<secret>\",\n}\nServiceUser is the Schema for the serviceusers API.
Exposes secret keys
SERVICEUSER_HOST, SERVICEUSER_PORT, SERVICEUSER_USERNAME, SERVICEUSER_PASSWORD, SERVICEUSER_CA_CERT, SERVICEUSER_ACCESS_CERT, SERVICEUSER_ACCESS_KEY.
Required
apiVersion (string). Value aiven.io/v1alpha1.kind (string). Value ServiceUser.metadata (object). Data that identifies the object, including a name string and optional namespace.spec (object). ServiceUserSpec defines the desired state of ServiceUser. See below for nested schema.Appears on ServiceUser.
ServiceUserSpec defines the desired state of ServiceUser.
Required
project (string, Immutable, Pattern: ^[a-zA-Z0-9_-]+$, MaxLength: 63). Identifies the project this resource belongs to.serviceName (string, Immutable, Pattern: ^[a-z][-a-z0-9]+$, MaxLength: 63). Specifies the name of the service that this resource belongs to.Optional
authSecretRef (object). Authentication reference to Aiven token in a secret. See below for nested schema.authentication (string, Enum: caching_sha2_password, mysql_native_password). Authentication details.connInfoSecretTarget (object). Secret configuration. See below for nested schema.connInfoSecretTargetDisabled (boolean, Immutable). When true, the secret containing connection information will not be created, defaults to false. This field cannot be changed after resource creation.Appears on spec.
Authentication reference to Aiven token in a secret.
Required
key (string, MinLength: 1).name (string, MinLength: 1).Appears on spec.
Secret configuration.
Required
name (string, Immutable). Name of the secret resource to be created. By default, it is equal to the resource name.Optional
annotations (object, AdditionalProperties: string). Annotations added to the secret.labels (object, AdditionalProperties: string). Labels added to the secret.prefix (string). Prefix for the secret's keys. Added \"as is\" without any transformations. By default, is equal to the kind name in uppercase + underscore, e.g. KAFKA_, REDIS_, etc.The Aiven Operator for Kubernetes project accepts contributions via GitHub pull requests. This document outlines the process to help get your contribution accepted.
Please see also the Aiven Operator for Kubernetes Developer Guide.
"},{"location":"contributing/index.html#support-channels","title":"Support Channels","text":"This project offers support through GitHub issues and can be filed here. Moreover, GitHub issues are used as the primary method for tracking anything to do with the Aiven Operator for Kubernetes project.
"},{"location":"contributing/index.html#pull-request-process","title":"Pull Request Process","text":"In the interest of fostering an open and welcoming environment, we as contributors and maintainers pledge to making participation in our project and our community a harassment-free experience for everyone, regardless of age, body size, disability, ethnicity, gender identity and expression, level of experience, nationality, personal appearance, race, religion, or sexual identity and orientation.
"},{"location":"contributing/index.html#our-standards","title":"Our Standards","text":"Examples of behavior that contributes to creating a positive environment include:
Examples of unacceptable behavior by participants include:
This project adheres to the Conventional Commits specification. Please, make sure that your commit messages follow that specification.
"},{"location":"contributing/index.html#our-responsibilities","title":"Our Responsibilities","text":"Project maintainers are responsible for clarifying the standards of acceptable behavior and are expected to take appropriate and fair corrective action in response to any instances of unacceptable behavior.
Project maintainers have the right and responsibility to remove, edit, or reject comments, commits, code, wiki edits, issues, and other contributions that are not aligned to this Code of Conduct, or to ban temporarily or permanently any contributor for other behaviors that they deem inappropriate, threatening, offensive, or harmful.
"},{"location":"contributing/index.html#scope","title":"Scope","text":"This Code of Conduct applies both within project spaces and in public spaces when an individual is representing the project or its community. Examples of representing a project or community include using an official project e-mail address, posting via an official social media account, or acting as an appointed representative at an online or offline event. Representation of a project may be further defined and clarified by project maintainers.
"},{"location":"contributing/index.html#enforcement","title":"Enforcement","text":"Instances of abusive, harassing, or otherwise unacceptable behavior may be reported by contacting the project team at opensource@aiven.io. All complaints will be reviewed and investigated and will result in a response that is deemed necessary and appropriate to the circumstances. The project team is obligated to maintain confidentiality with regard to the reporter of an incident. Further details of specific enforcement policies may be posted separately.
Project maintainers who do not follow or enforce the Code of Conduct in good faith may face temporary or permanent repercussions as determined by other members of the project's leadership.
"},{"location":"contributing/developer-guide.html","title":"Developer guide","text":""},{"location":"contributing/developer-guide.html#get-started","title":"Get Started","text":"To get started, make sure you have a working Go environment and clone the repository:
git clone git@github.com:aiven/aiven-operator.git\ncd aiven-operator\nBefore proceeding, please ensure that you have the following dependencies installed:
-w0 flag, which may cause some tests to not work properly)Create an Aiven authentication token and an Aiven project, you'll need them in the following sections:
AIVEN_TOKEN \u2014 your authentication tokenAIVEN_PROJECT_NAME \u2014 the name of you Aiven project to run the services inOptional dependencies:
The project uses the make build system. To see the available targets, run make help.
Building the operator binary:
make build\nTo run the operator locally, first create a local k8s cluster with kind:
$ kind create cluster --image kindest/node:v1.26.6 --wait 5m\nCreating cluster \"kind\" ...\n \u2713 Ensuring node image (kindest/node:v1.26.6) \ud83d\uddbc\n \u2713 Preparing nodes \ud83d\udce6\n \u2713 Writing configuration \ud83d\udcdc\n \u2713 Starting control-plane \ud83d\udd79\ufe0f\n \u2713 Installing CNI \ud83d\udd0c\n \u2713 Installing StorageClass \ud83d\udcbe\n \u2713 Waiting \u2264 5m0s for control-plane = Ready \u23f3\n \u2022 Ready after 14s \ud83d\udc9a\nSet kubectl context to \"kind-kind\"\nYou can now use your cluster with:\n\nkubectl cluster-info --context kind-kinds\nThis creates a k8s cluster and points your kubectl to that cluster:
$ kubectl cluster-info --context kind-kind\nKubernetes control plane is running at https://127.0.0.1:58083\nCoreDNS is running at https://127.0.0.1:58083/api/v1/namespaces/kube-system/services/kube-dns:dns/proxy\nRun the operator:
ENABLE_WEBHOOKS=false make run\nIn another terminal window, create a Kubernetes Secret named aiven-token (use the Aiven authentication token created earlier):
export AIVEN_TOKEN=...\nkubectl create secret generic aiven-token --from-literal=token=$AIVEN_TOKEN\nThen, create a manifest for an Aiven PostgreSQL service:
aiven-pg.yamlapiVersion: aiven.io/v1alpha1\nkind: PostgreSQL\nmetadata:\n name: aiven-devdocs-pg\nspec:\n # reads the authentication token\n authSecretRef:\n name: aiven-token\n key: token\n\n # stores the PostgreSQL connection information on the specified Secret\n connInfoSecretTarget:\n name: pg-connection\n\n project: <your-project-name>\n cloudName: google-europe-west1\n plan: hobbyist\n maintenanceWindowDow: friday\n maintenanceWindowTime: 23:00:00\n userConfig:\n pg_version: \"15\"\n...and create the service at Aiven by applying the manifest:
kubectl apply -f aiven-pg.yaml\nYou should see the service be created on Aiven's side using either Aiven Console or Aiven CLI:
$ avn project switch PROJECT_NAME\n$ avn service list\nSERVICE_NAME SERVICE_TYPE STATE CLOUD_NAME PLAN CREATE_TIME UPDATE_TIME NOTIFICATIONS\n================ ============ ========== =================== ======== ==================== ==================== =============\naiven-devdocs-pg pg REBUILDING google-europe-west1 hobbyist 2024-01-01T00:00:00Z 2024-01-01T00:01:00Z\nTo see the resource on k8s side, use kubectl:
$ kubectl get postgresqls\nNAME PROJECT REGION PLAN STATE\naiven-devdocs-pg aiven-sbox-k8s-riski google-europe-west1 hobbyist REBUILDING\n... or by running k9s in another terminal window and type \": postgresqls\" to see the provisioned PostgreSQL resources:
For this section you'll need to set export AIVEN_TOKEN=... and an export AIVEN_PROJECT_NAME=... as well as export ENABLE_WEBHOOKS=false (see Prerequisites).
Note
If you encounter any issues with webhooks, you have the option to disable them by running export ENABLE_WEBOOKS=false
Running all unit tests is possible by running make test. To run a targeted test, use run=TestGrafana make test.
Setup the test environment:
make e2e-setup-kind\nThe e2e tests are configured to run against a project called aiven-ci-kubernetes-operator. To run the test against
make test-e2e-preinstalled\nTo remove the kind cluster, simply execute the following command:
kind delete cluster\nPlease see this page for more information.
"},{"location":"contributing/developer-guide.html#documentation","title":"Documentation","text":"The documentation is written in markdown and generated by mkdocs and mkdocs-material.
To run the documentation live preview:
make serve-docs\nAnd open the http://localhost:8000/aiven-operator/ page in your web browser.
The documentation API Reference section is generated automatically from the source code during the documentation deployment. To generate it locally, run the following command:
make docs\nAiven Kubernetes Operator generates service configs code (also known as user configs) and documentation from public service types schema.
"},{"location":"contributing/resource-generation.html#flow-overview","title":"Flow overview","text":"When a new schema is issued on the API, a cron job fetches it, parses, patches, and saves in a shared library \u2014 go-api-schemas.
When the library is updated, the GitHub dependabot creates PRs to the dependent repositories, like Aiven Kubernetes Operator and Aiven Terraform Provider.
Then the make generate command is called by GitHub action. And the PR is ready for review.
flowchart TB\n API(Aiven API) <-.->|polls schema updates| Schema([go-api-schemas])\n Bot(dependabot) <-.->|polls updates| Schema\n Bot-->|pull request|UpdateOP[/\"\u2728 $ make generate \u2728\"/]\n UpdateOP-->|review| OP([operator repository])The command runs several generators in a certain sequence. First, the user config generator is called. Then controller-gen cli. Then API reference docs generator and charts generator.
Here how it goes in the details:
./<api-reference-docs>/example/, if it finds one, it validates that with the CRD. Each CRD has an OpenAPI v3 schema as a part of it. This is also used by Kubernetes itself to validate user input.flowchart TB\n Make[/$ make generate/]-->Generator(userconfig generator<br> creates/updates structs using updated spec)\n Generator-->|go: KafkaUserConfig struct| K8S(controller-gen<br> adds k8s methods to structs)\n K8S-->|go files| CRD(controller-gen<br> creates CRDs out of structs)\n CRD-->|CRD: aiven.io_kafkas.yaml| Docs(docs generator)\n subgraph API reference generation\n Docs-->|aiven.io_kafkas.yaml|Reference(creates reference<br> out of CRD)\n Docs-->|examples/kafka.yaml,<br> aiven.io_kafkas.yaml|Examples(validates example<br> using CRD)\n Examples--> Markdown(creates docs out of CRDs, adds examples)\n Reference-->Markdown(kafka.md)\n end\n CRD-->|yaml files|Charts(charts generator<br> updates helm charts<br> and the changelog)\n Charts-->ToRelease(\"Ready to release \ud83c\udf89\")\n Markdown-->ToReleaseBy default, charts generator keeps the current helm chart's version, because it doesn't know semver. You need it to do manually.
To do so run the following command with the version of your choice:
make version=v1.0.0 charts\nThe Aiven Operator for Kubernetes can be installed via Helm.
Before you start, make sure you have the prerequisites.
First add the Aiven Helm repository:
helm repo add aiven https://aiven.github.io/aiven-charts && helm repo update\nhelm install aiven-operator-crds aiven/aiven-operator-crds\nVerify the installation:
kubectl api-resources --api-group=aiven.io\nThe output is similar to the following:
NAME SHORTNAMES APIVERSION NAMESPACED KIND\nconnectionpools aiven.io/v1alpha1 true ConnectionPool\ndatabases aiven.io/v1alpha1 true Database\n... < several omitted lines >\nhelm install aiven-operator aiven/aiven-operator\nNote
Installation will fail if webhooks are enabled and the CRDs for the cert-manager are not installed.
Verify the installation:
helm status aiven-operator\nThe output is similar to the following:
NAME: aiven-operator\nLAST DEPLOYED: Fri Sep 10 15:23:26 2021\nNAMESPACE: default\nSTATUS: deployed\nREVISION: 1\nTEST SUITE: None\nIt is also possible to install the operator without webhooks enabled:
helm install aiven-operator aiven/aiven-operator --set webhooks.enabled=false\nPlease refer to the values.yaml of the chart.
"},{"location":"installation/helm.html#install-without-full-cluster-administrator-access","title":"Install without full cluster administrator access","text":"If the person installing the Helm chart does not have the necessary permissions to create cluster-wide resources such as ClusterRole and ClusterRoleBinding, a cluster administrator can manually install these roles. This ensures that the operator can function properly.
Important
Please see this page for more information.
Find out the name of your deployment:
helm list\nThe output has the name of each deployment similar to the following:
NAME NAMESPACE REVISION UPDATED STATUS CHART APP VERSION\naiven-operator default 1 2021-09-09 10:56:14.623700249 +0200 CEST deployed aiven-operator-v0.1.0 v0.1.0\naiven-operator-crds default 1 2021-09-09 10:56:05.736411868 +0200 CEST deployed aiven-operator-crds-v0.1.0 v0.1.0\nRemove the CRDs:
helm uninstall aiven-operator-crds\nThe confirmation message is similar to the following:
release \"aiven-operator-crds\" uninstalled\nRemove the operator:
helm uninstall aiven-operator\nThe confirmation message is similar to the following:
release \"aiven-operator\" uninstalled\nBefore you start, make sure you have the prerequisites.
All Aiven Operator for Kubernetes components can be installed from one YAML file that is uploaded for every release:
kubectl apply -f https://github.com/aiven/aiven-operator/releases/latest/download/deployment.yaml\nBy default the Deployment is installed into the aiven-operator-system namespace.
Assuming you installed version vX.Y.Z of the operator it can be uninstalled via
kubectl delete -f https://github.com/aiven/aiven-operator/releases/download/vX.Y.Z/deployment.yaml\nThe Aiven Operator for Kubernetes supports all major Kubernetes distributions, both locally and in the cloud.
Make sure you have the following:
The Aiven Operator for Kubernetes uses cert-manager to configure the service reference of our webhooks.
Please follow the installation instructions on their website.
Note
This is not required in the Helm installation if you select to disable webhooks, but that is not recommended outside of playground use.
The Aiven Operator for Kubernetes uses webhooks for setting defaults and enforcing invariants that are expected by the aiven API and will lead to errors if ignored.
In the future webhooks will also be used for conversion and supporting multiple CRD versions.
"},{"location":"installation/uninstalling.html","title":"Uninstall","text":"Danger
Uninstalling the Aiven Operator for Kubernetes can remove the resources created in Aiven, possibly resulting in data loss.
Depending on your installation, please follow one of:
Aiven resources need to have an accompanying secret that contains the token that is used to authorize the manipulation of that resource. If that token expired then you will not be able to delete the custom resource and deletion will also hang until the situation is resolved. The recommended approach to deal with that situation is to patch a valid token into the secret again so that proper cleanup of aiven resources can take place.
"},{"location":"installation/uninstalling.html#hanging-deletions","title":"Hanging deletions","text":"To protect the Secrets that the operator is using from deletion, it adds the finalizer finalizers.aiven.io/needed-to-delete-services to the Secret. This solves a race condition that happens when deleting a namespace, where there is a possibility of the Secret getting deleted before the resource that uses it. When the controller is deleted it may not cleanup the finalizers from all Secrets. If there is a Secret with this finalizer blocking deletion of a namespace, you can remove the finalizer by running:
kubectl patch secret <offending-secret> -p '{\"metadata\":{\"finalizers\":null}}' --type=merge\nAiven for Apache Cassandra\u00ae is a distributed database designed to handle large volumes of writes.
Note
Before going through this guide, make sure you have a Kubernetes cluster with the operator installed (see instructions for helm or kubectl), and a Kubernetes Secret with an Aiven authentication token.
"},{"location":"resources/cassandra.html#create-a-cassandra-instance","title":"Create a Cassandra instance","text":"1. Create a file named cassandra-sample.yaml, and add the following content:
apiVersion: aiven.io/v1alpha1\nkind: Cassandra\nmetadata:\n name: cassandra-sample\nspec:\n # gets the authentication token from the `aiven-token` Secret\n authSecretRef:\n name: aiven-token\n key: token\n\n # outputs the Cassandra connection on the `cassandra-secret` Secret\n connInfoSecretTarget:\n name: cassandra-secret\n\n # add your Project name here\n project: PROJECT_NAME\n\n # cloud provider and plan of your choice\n # you can check all of the possibilities here https://aiven.io/pricing\n cloudName: google-europe-west1\n plan: startup-4\n\n # general Aiven configuration\n maintenanceWindowDow: friday\n maintenanceWindowTime: 23:00:00\n2. Create the service by applying the configuration:
kubectl apply -f cassandra-sample.yaml\nThe output is:
cassandra.aiven.io/cassandra-sample created\n3. Review the resource you created with this command:
kubectl describe cassandra.aiven.io cassandra-sample\nThe output is similar to the following:
...\nStatus:\n Conditions:\n Last Transition Time: 2023-01-31T10:17:25Z\n Message: Successfully created or updated the instance in Aiven\n Reason: Created\n Status: True\n Type: Initialized\n Last Transition Time: 2023-01-31T10:24:00Z\n Message: Instance is running on Aiven side\n Reason: CheckRunning\n Status: True\n Type: Running\n State: RUNNING\n...\nThe resource can be in the REBUILDING state for a few minutes. Once the state changes to RUNNING, you can access the resource.
For your convenience, the operator automatically stores the Cassandra connection information in a Secret created with the name specified on the connInfoSecretTarget field.
To view the details of the Secret, use the following command:
kubectl describe secret cassandra-secret\nThe output is similar to the following:
Name: cassandra-secret\nNamespace: default\nLabels: <none>\nAnnotations: <none>\n\nType: Opaque\n\nData\n====\nCASSANDRA_HOSTS: 59 bytes\nCASSANDRA_PASSWORD: 24 bytes\nCASSANDRA_PORT: 5 bytes\nCASSANDRA_URI: 66 bytes\nCASSANDRA_USER: 8 bytes\nCASSANDRA_HOST: 60 bytes\nYou can use the jq to quickly decode the Secret:
kubectl get secret cassandra-secret -o json | jq '.data | map_values(@base64d)'\nThe output is similar to the following:
{\n \"CASSANDRA_HOST\": \"<secret>\",\n \"CASSANDRA_HOSTS\": \"<secret>\",\n \"CASSANDRA_PASSWORD\": \"<secret>\",\n \"CASSANDRA_PORT\": \"14609\",\n \"CASSANDRA_URI\": \"<secret>\",\n \"CASSANDRA_USER\": \"avnadmin\"\n}\nYou can create service users for your instance of Aiven for Apache Cassandra. Service users are unique to this instance and are not shared with any other services.
1. Create a file named cassandra-service-user.yaml:
apiVersion: aiven.io/v1alpha1\nkind: ServiceUser\nmetadata:\n name: cassandra-service-user\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n connInfoSecretTarget:\n name: cassandra-service-user-secret\n\n project: PROJECT_NAME\n serviceName: cassandra-sample\n2. Create the user by applying the configuration:
kubectl apply -f cassandra-service-user.yaml\nThe ServiceUser resource generates a Secret with connection information.
3. View the details of the Secret using the following command:
kubectl get secret cassandra-service-user-secret -o json | jq '.data | map_values(@base64d)'\nThe output is similar to the following:
{\n \"ACCESS_CERT\": \"<secret>\",\n \"ACCESS_KEY\": \"<secret>\",\n \"CA_CERT\": \"<secret>\",\n \"HOST\": \"<secret>\",\n \"PASSWORD\": \"<secret>\",\n \"PORT\": \"14609\",\n \"USERNAME\": \"cassandra-service-user\"\n}\nYou can connect to the Cassandra instance using these credentials and the host information from the cassandra-secret Secret.
Aiven for ClickHouse\u00ae is a fully managed distributed columnar database based on open source ClickHouse.
Note
Before going through this guide, make sure you have a Kubernetes cluster with the operator installed (see instructions for helm or kubectl), and a Kubernetes Secret with an Aiven authentication token.
"},{"location":"resources/clickhouse.html#create-a-clickhouse-instance","title":"Create a ClickHouse instance","text":"1. Create a file named clickhouse-sample.yaml, and add the following:
apiVersion: aiven.io/v1alpha1\nkind: Clickhouse\nmetadata:\n name: example-clickhouse\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n # Outputs the ClickHouse connection to the `clickhouse-secret` Secret.\n connInfoSecretTarget:\n name: clickhouse-secret\n # Your Aiven project.\n project: PROJECT_NAME\n # Choose a cloud provider and plan.\n # View the options on the pricing page at https://aiven.io/pricing.\n cloudName: google-europe-west1\n plan: startup-16\n # Configure the maintenance window.\n maintenanceWindowDow: friday\n maintenanceWindowTime: 23:00:00\nPROJECT_NAME is the name of your Aiven project. 2. To create the service, run:
kubectl apply -f clickhouse-sample.yaml\nThe output is similar to the following:
clickhouse.aiven.io/example-clickhouse created\n3. To review the resource you created, run:
kubectl describe clickhouse.aiven.io example-clickhouse\nThe output is similar to the following:
...\nStatus:\n Conditions:\n Last Transition Time: 2024-06-25T07:58:37Z\n Message: Instance was created or update on Aiven side\n Reason: Created\n Status: True\n Type: Initialized\n Last Transition Time: 2024-06-25T08:01:47Z\n Message: Instance is running on Aiven side\n Reason: CheckRunning\n Status: True\n Type: Running\n State: RUNNING\nEvents:\n Type Reason Age From Message\n ---- ------ ---- ---- -------\n Normal CreatedOrUpdatedAtAiven 3m24s (x5 over 3m45s) clickhouse-reconciler waiting for the instance to be running\n Normal ReconcilationStarted 3m14s (x6 over 3m47s) clickhouse-reconciler starting reconciliation\n Normal InstanceFinalizerAdded 3m14s (x6 over 3m47s) clickhouse-reconciler waiting for preconditions of the instance\n...\nREBUILDING state for a few minutes. Once the state changes to RUNNING, you can access the service."},{"location":"resources/clickhouse.html#use-the-connection-secret","title":"Use the connection Secret","text":"Aiven Operator automatically stores the ClickHouse connection information in a Secret created with the name in the connInfoSecretTarget field.
To view the details of the Secret, run:
kubectl describe secret clickhouse-secret\nName: clickhouse-secret\nNamespace: default\nLabels: <none>\nAnnotations: <none>\n\nType: Opaque\n\nData\n====\nCLICKHOUSE_USER: 8 bytes\nHOST: 46 bytes\nPASSWORD: 24 bytes\nPORT: 5 bytes\nUSER: 8 bytes\nCLICKHOUSE_HOST: 46 bytes\nCLICKHOUSE_PASSWORD: 24 bytes\nCLICKHOUSE_PORT: 5 bytes\nYou can use the JSON processor jq to decode the Secret:
kubectl get secret clickhouse-secret -o json | jq '.data | map_values(@base64d)'\nThe output is similar to the following:
{\n \"CLICKHOUSE_HOST\": \"HOST\",\n \"CLICKHOUSE_PASSWORD\": \"SERVICE_PASSWORD\",\n \"CLICKHOUSE_PORT\": \"12691\",\n \"CLICKHOUSE_USER\": \"avnadmin\",\n \"HOST\": \"HOST\",\n \"PASSWORD\": \"ADMIN_USER_PASSWORD\",\n \"PORT\": \"12691\",\n \"USER\": \"avnadmin\"\n}\nNote
Tables cannot be created using Aiven Operator. To create a table, use the Aiven Console or CLI.
1. Create a file named clickhouse-db.yaml and add the following:
apiVersion: aiven.io/v1alpha1\nkind: ClickhouseDatabase\nmetadata:\n name: example-database\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n serviceName: example-clickhouse\n project: PROJECT_NAME\nWhere PROJECT_NAME is the name of your Aiven project.
2. To create the database, run:
kubectl apply -f clickhouse-db.yaml\n3. To view details about the resource, run:
kubectl describe ClickhouseDatabase example-database\nThe output is similar to the following:
...\nSpec:\n Auth Secret Ref:\n Key: token\n Name: aiven-token\n Project: example-project\n Service Name: example-clickhouse\nStatus:\n Conditions:\n Last Transition Time: 2024-06-25T13:58:35Z\n Message: Checking preconditions\n Reason: Preconditions\n Status: True\n Type: Initialized\n Last Transition Time: 2024-06-25T14:32:05Z\n Message: Instance is running on Aiven side\n Reason: CheckRunning\n Status: True\n Type: Running\n...\nYou can create service users and roles for an instance of ClickHouse, and grant privileges to them. Users and roles are not shared with any other services.
1. Create a file named clickhouse-service-users.yaml and add the following configuration for a service user:
apiVersion: aiven.io/v1alpha1\nkind: ClickhouseUser\nmetadata:\n name: example-user\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n connInfoSecretTarget:\n name: clickhouse-service-user-secret\n serviceName: example-clickhouse\n project: PROJECT_NAME\nPROJECT_NAME is the name of your Aiven project. This resource generates a Secret with connection information and stores it in clickhouse-service-user-secret.
2. To create a role add the following to the same file:
---\n\napiVersion: aiven.io/v1alpha1\nkind: ClickhouseRole\nmetadata:\n name: example-role\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n serviceName: example-clickhouse\n project: PROJECT_NAME\n role: read-only\n3. Grant privileges to the role, and assign the role to the user by adding the following to the same file:
---\n\napiVersion: aiven.io/v1alpha1\nkind: ClickhouseGrant\nmetadata:\n name: example-grant\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n project: PROJECT_NAME\n serviceName: example-clickhouse\n\n privilegeGrants:\n - grantees:\n - role: read-only\n privileges:\n - SELECT\n database: example-database\n\n roleGrants:\n - roles:\n - read-only\n grantees:\n - user: example-user\n4. To create the user, role, and grant, run:
kubectl apply -f clickhouse-service-users.yaml\n5. To get credentials and host information for connecting to this instance, view the connection information stored in the Secret by running:
kubectl get secret clickhouse-service-user-secret -o json | jq '.data | map_values(@base64d)'\nThe output is similar to the following:
{\n \"CLICKHOUSEUSER_HOST\": \"HOST\",\n \"CLICKHOUSEUSER_PASSWORD\": \"PASSWORD\",\n \"CLICKHOUSEUSER_PORT\": \"12691\",\n \"CLICKHOUSEUSER_USERNAME\": \"example-user\",\n \"HOST\": \"HOST\",\n \"PASSWORD\": \"PASSWORD\",\n \"PORT\": \"12691\",\n \"USERNAME\": \"example-user\"\n}\nAiven for MySQL is a fully managed relational database service, deployable in the cloud of your choice.
Before going through this guide, make sure you have a Kubernetes cluster with the operator installed (see instructions for helm or kubectl) and a Kubernetes Secret with an Aiven authentication token.
"},{"location":"resources/mysql.html#create-a-mysql-instance","title":"Create a MySQL instance","text":"1. Create a file named mysql-sample.yaml, and add the following content:
apiVersion: aiven.io/v1alpha1\nkind: MySQL\nmetadata:\n name: mysql-sample\nspec:\n # gets the authentication token from the `aiven-token` Secret\n authSecretRef:\n name: aiven-token\n key: token\n\n # outputs the MySQL connection on the `mysql-secret` Secret\n connInfoSecretTarget:\n name: mysql-secret\n\n # add your Project name here\n project: PROJECT_NAME\n\n # cloud provider and plan of your choice\n # you can check all of the possibilities here https://aiven.io/pricing\n cloudName: google-europe-west1\n plan: startup-4\n\n # general Aiven configuration\n maintenanceWindowDow: friday\n maintenanceWindowTime: 23:00:00\n2. Create the service by applying the configuration:
kubectl apply -f mysql-sample.yaml\n3. Review the resource you created with this command:
kubectl describe mysql.aiven.io mysql-sample\nThe output is similar to the following:
...\nStatus:\n Conditions:\n Last Transition Time: 2023-02-22T15:43:44Z\n Message: Successfully created or updated the instance in Aiven\n Reason: Created\n Status: True\n Type: Initialized\n Last Transition Time: 2023-02-22T15:43:44Z\n Message: Successfully created or updated the instance in Aiven, status remains unknown\n Reason: Created\n Status: Unknown\n Type: Running\n State: REBUILDING\n...\nThe resource will be in the REBUILDING state for a few minutes. Once the state changes to RUNNING, you can access the resource.
For your convenience, the operator automatically stores the MySQL connection information in a Secret created with the name specified on the connInfoSecretTarget field.
To view the details of the Secret, use the following command:
kubectl describe secret mysql-secret\nThe output is similar to the following:
Name: mysql-secret\nNamespace: default\nLabels: <none>\nAnnotations: <none>\n\nType: Opaque\n\nData\n====\nMYSQL_PORT: 5 bytes\nMYSQL_SSL_MODE: 8 bytes\nMYSQL_URI: 115 bytes\nMYSQL_USER: 8 bytes\nMYSQL_DATABASE: 9 bytes\nMYSQL_HOST: 39 bytes\nMYSQL_PASSWORD: 24 bytes\nYou can use jq to quickly decode the Secret:
kubectl get secret mysql-secret -o json | jq '.data | map_values(@base64d)'\nThe output is similar to the following:
{\n \"MYSQL_DATABASE\": \"defaultdb\",\n \"MYSQL_HOST\": \"<secret>\",\n \"MYSQL_PASSWORD\": \"<secret>\",\n \"MYSQL_PORT\": \"12691\",\n \"MYSQL_SSL_MODE\": \"REQUIRED\",\n \"MYSQL_URI\": \"<secret>\",\n \"MYSQL_USER\": \"avnadmin\"\n}\nYou can create service users for your instance of Aiven for MySQL. Service users are unique to this instance and are not shared with any other services.
1. Create a file named mysql-service-user.yaml:
apiVersion: aiven.io/v1alpha1\nkind: ServiceUser\nmetadata:\n name: mysql-service-user\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n connInfoSecretTarget:\n name: mysql-service-user-secret\n\n project: PROJECT_NAME\n serviceName: mysql-sample\n2. Create the user by applying the configuration:
kubectl apply -f mysql-service-user.yaml\nThe ServiceUser resource generates a Secret with connection information.
3. View the details of the Secret using jq:
kubectl get secret mysql-service-user-secret -o json | jq '.data | map_values(@base64d)'\nThe output is similar to the following:
{\n \"ACCESS_CERT\": \"<secret>\",\n \"ACCESS_KEY\": \"<secret>\",\n \"CA_CERT\": \"<secret>\",\n \"HOST\": \"<secret>\",\n \"PASSWORD\": \"<secret>\",\n \"PORT\": \"14609\",\n \"USERNAME\": \"mysql-service-user\"\n}\nYou can connect to the MySQL instance using these credentials and the host information from the mysql-secret Secret.
OpenSearch\u00ae is an open source search and analytics suite including search engine, NoSQL document database, and visualization interface. OpenSearch offers a distributed, full-text search engine based on Apache Lucene\u00ae with a RESTful API interface and support for JSON documents.
Note
Before going through this guide, make sure you have a Kubernetes cluster with the operator installed (see instructions for helm or kubectl), and a Kubernetes Secret with an Aiven authentication token.
"},{"location":"resources/opensearch.html#create-an-opensearch-instance","title":"Create an OpenSearch instance","text":"1. Create a file named os-sample.yaml, and add the following content:
apiVersion: aiven.io/v1alpha1\nkind: OpenSearch\nmetadata:\n name: os-sample\nspec:\n # gets the authentication token from the `aiven-token` Secret\n authSecretRef:\n name: aiven-token\n key: token\n\n # outputs the OpenSearch connection on the `os-secret` Secret\n connInfoSecretTarget:\n name: os-secret\n\n # add your Project name here\n project: PROJECT_NAME\n\n # cloud provider and plan of your choice\n # you can check all of the possibilities here https://aiven.io/pricing\n cloudName: google-europe-west1\n plan: startup-4\n\n # general Aiven configuration\n maintenanceWindowDow: friday\n maintenanceWindowTime: 23:00:00\n2. Create the service by applying the configuration:
kubectl apply -f os-sample.yaml\n3. Review the resource you created with this command:
kubectl describe opensearch.aiven.io os-sample\nThe output is similar to the following:
...\nStatus:\n Conditions:\n Last Transition Time: 2023-01-19T14:41:43Z\n Message: Successfully created or updated the instance in Aiven\n Reason: Created\n Status: True\n Type: Initialized\n Last Transition Time: 2023-01-19T14:41:43Z\n Message: Successfully created or updated the instance in Aiven, status remains unknown\n Reason: Created\n Status: Unknown\n Type: Running\n State: REBUILDING\n...\nThe resource will be in the REBUILDING state for a few minutes. Once the state changes to RUNNING, you can access the resource.
For your convenience, the operator automatically stores the OpenSearch connection information in a Secret created with the name specified on the connInfoSecretTarget field.
To view the details of the Secret, use the following command:
kubectl describe secret os-secret\nThe output is similar to the following:
Name: os-secret\nNamespace: default\nLabels: <none>\nAnnotations: <none>\n\nType: Opaque\n\nData\n====\nHOST: 61 bytes\nPASSWORD: 24 bytes\nPORT: 5 bytes\nUSER: 8 bytes\nYou can use the jq to quickly decode the Secret:
kubectl get secret os-secret -o json | jq '.data | map_values(@base64d)'\nThe output is similar to the following:
{\n \"HOST\": \"os-sample-your-project.aivencloud.com\",\n \"PASSWORD\": \"<secret>\",\n \"PORT\": \"13041\",\n \"USER\": \"avnadmin\"\n}\nYou can create service users for your instance of Aiven for OpenSearch. Service users are unique to this instance and are not shared with any other services.
1. Create a file named os-service-user.yaml:
apiVersion: aiven.io/v1alpha1\nkind: ServiceUser\nmetadata:\n name: os-service-user\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n connInfoSecretTarget:\n name: os-service-user-secret\n\n project: PROJECT_NAME\n serviceName: os-sample\n2. Create the user by applying the configuration:
kubectl apply -f os-service-user.yaml\nThe ServiceUser resource generates a Secret with connection information.
3. View the details of the Secret using the following command:
kubectl get secret os-service-user-secret -o json | jq '.data | map_values(@base64d)'\nThe output is similar to the following:
{\n \"ACCESS_CERT\": \"<secret>\",\n \"ACCESS_KEY\": \"<secret>\",\n \"CA_CERT\": \"<secret>\",\n \"HOST\": \"os-sample-your-project.aivencloud.com\",\n \"PASSWORD\": \"<secret>\",\n \"PORT\": \"14609\",\n \"USERNAME\": \"os-service-user\"\n}\nYou can connect to the OpenSearch instance using these credentials and the host information from the os-secret Secret.
PostgreSQL is an open source, relational database. It's ideal for organisations that need a well organised tabular datastore. On top of the strict table and columns formats, PostgreSQL also offers solutions for nested datasets with the native jsonb format and advanced set of extensions including PostGIS, a spatial database extender for location queries. Aiven for PostgreSQL is the perfect fit for your relational data.
With Aiven Kubernetes Operator, you can manage Aiven for PostgreSQL through the well defined Kubernetes API.
Note
Before going through this guide, make sure you have a Kubernetes cluster with the operator installed (see instructions for helm or kubectl), and a Kubernetes Secret with an Aiven authentication token.
"},{"location":"resources/postgresql.html#create-a-postgresql-instance","title":"Create a PostgreSQL instance","text":"1. Create a file named pg-sample.yaml with the following content:
apiVersion: aiven.io/v1alpha1\nkind: PostgreSQL\nmetadata:\n name: pg-sample\nspec:\n # gets the authentication token from the `aiven-token` Secret\n authSecretRef:\n name: aiven-token\n key: token\n\n # outputs the PostgreSQL connection on the `pg-connection` Secret\n connInfoSecretTarget:\n name: pg-connection\n\n # add your Project name here\n project: PROJECT_NAME\n\n # cloud provider and plan of your choice\n # you can check all of the possibilities here https://aiven.io/pricing\n cloudName: google-europe-west1\n plan: startup-4\n\n # general Aiven configuration\n maintenanceWindowDow: friday\n maintenanceWindowTime: 23:00:00\n\n # specific PostgreSQL configuration\n userConfig:\n pg_version: \"11\"\n2. Create the service by applying the configuration:
kubectl apply -f pg-sample.yaml\n3. Review the resource you created with the following command:
kubectl get postgresqls.aiven.io pg-sample\nThe output is similar to the following:
NAME PROJECT REGION PLAN STATE\npg-sample your-project google-europe-west1 startup-4 RUNNING\nThe resource can stay in the BUILDING state for a couple of minutes. Once the state changes to RUNNING, you are ready to access it.
For your convenience, the operator automatically stores the PostgreSQL connection information in a Secret created with the name specified on the connInfoSecretTarget field.
kubectl describe secret pg-connection\nThe output is similar to the following:
Name: pg-connection\nNamespace: default\nAnnotations: <none>\n\nType: Opaque\n\nData\n====\nDATABASE_URI: 107 bytes\nPGDATABASE: 9 bytes\nPGHOST: 38 bytes\nPGPASSWORD: 16 bytes\nPGPORT: 5 bytes\nPGSSLMODE: 7 bytes\nPGUSER: 8 bytes\nYou can use the jq to quickly decode the Secret:
kubectl get secret pg-connection -o json | jq '.data | map_values(@base64d)'\nThe output is similar to the following:
{\n \"DATABASE_URI\": \"postgres://avnadmin:<secret-password>@pg-sample-your-project.aivencloud.com:13039/defaultdb?sslmode=require\",\n \"PGDATABASE\": \"defaultdb\",\n \"PGHOST\": \"pg-sample-your-project.aivencloud.com\",\n \"PGPASSWORD\": \"<secret-password>\",\n \"PGPORT\": \"13039\",\n \"PGSSLMODE\": \"require\",\n \"PGUSER\": \"avnadmin\"\n}\nYou can verify your PostgreSQL connection from a Kubernetes workload by deploying a Pod that runs the psql command.
1. Create a file named pod-psql.yaml
apiVersion: v1\nkind: Pod\nmetadata:\n name: psql-test-connection\nspec:\n restartPolicy: Never\n containers:\n - image: postgres:11-alpine\n name: postgres\n command: [\"psql\", \"$(DATABASE_URI)\", \"-c\", \"SELECT version();\"]\n\n # the pg-connection Secret becomes environment variables\n envFrom:\n - secretRef:\n name: pg-connection\nIt runs once and stops, due to the restartPolicy: Never flag.
2. Inspect the log:
kubectl logs psql-test-connection\nThe output is similar to the following:
version\n---------------------------------------------------------------------------------------------\n PostgreSQL 11.12 on x86_64-pc-linux-gnu, compiled by gcc, a 68c5366192 p 6b9244f01a, 64-bit\n(1 row)\nYou have now connected to the PostgreSQL, and executed the SELECT version(); query.
The Database Kubernetes resource allows you to create a logical database within the PostgreSQL instance.
Create the pg-database-sample.yaml file with the following content:
apiVersion: aiven.io/v1alpha1\nkind: Database\nmetadata:\n name: pg-database-sample\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n # the name of the previously created PostgreSQL instance\n serviceName: pg-sample\n\n project: PROJECT_NAME\n lcCollate: en_US.UTF-8\n lcCtype: en_US.UTF-8\nYou can now connect to the pg-database-sample using the credentials stored in the pg-connection Secret.
Aiven uses the concept of service user that allows you to create users for different services. You can create one for the PostgreSQL instance.
1. Create a file named pg-service-user.yaml.
apiVersion: aiven.io/v1alpha1\nkind: ServiceUser\nmetadata:\n name: pg-service-user\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n connInfoSecretTarget:\n name: pg-service-user-connection\n\n project: PROJECT_NAME\n serviceName: pg-sample\n2. Apply the configuration with the following command.
kubectl apply -f pg-service-user.yaml\nThe ServiceUser resource generates a Secret with connection information, in this case named pg-service-user-connection:
kubectl get secret pg-service-user-connection -o json | jq '.data | map_values(@base64d)'\nThe output has the password and username:
{\n \"PASSWORD\": \"<secret-password>\",\n \"USERNAME\": \"pg-service-user\"\n}\nYou can now connect to the PostgreSQL instance using the credentials generated above, and the host information from the pg-connection Secret.
Connection pooling allows you to maintain very large numbers of connections to a database while minimizing the consumption of server resources. For more information, refer to the connection pooling article in Aiven Docs. Aiven for PostgreSQL uses PGBouncer for connection pooling.
You can create a connection pool with the ConnectionPool resource using the previously created Database and ServiceUser:
Create a new file named pg-connection-pool.yaml with the following content:
apiVersion: aiven.io/v1alpha1\nkind: ConnectionPool\nmetadata:\n name: pg-connection-pool\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n connInfoSecretTarget:\n name: pg-connection-pool-connection\n\n project: PROJECT_NAME\n serviceName: pg-sample\n databaseName: pg-database-sample\n username: pg-service-user\n poolSize: 10\n poolMode: transaction\nThe ConnectionPool generates a Secret with the connection info using the name from the connInfoSecretTarget.Name field:
kubectl get secret pg-connection-pool-connection -o json | jq '.data | map_values(@base64d)'\nThe output is similar to the following:
{\n \"DATABASE_URI\": \"postgres://pg-service-user:<secret-password>@pg-sample-you-project.aivencloud.com:13040/pg-connection-pool?sslmode=require\",\n \"PGDATABASE\": \"pg-database-sample\",\n \"PGHOST\": \"pg-sample-your-project.aivencloud.com\",\n \"PGPASSWORD\": \"<secret-password>\",\n \"PGPORT\": \"13040\",\n \"PGSSLMODE\": \"require\",\n \"PGUSER\": \"pg-service-user\"\n}\nRead-only replicas can be used to reduce the load on the primary service by making read-only queries against the replica service.
To create a read-only replica for a PostgreSQL service, you create a second PostgreSQL service and use serviceIntegrations to replicate data from your primary service.
The example that follows creates a primary service and a read-only replica.
1. Create a new file named pg-read-replica.yaml with the following:
apiVersion: aiven.io/v1alpha1\nkind: PostgreSQL\nmetadata:\n name: primary-pg-service\nspec:\n # gets the authentication token from the `aiven-token` Secret\n authSecretRef:\n name: aiven-token\n key: token\n\n # add your project's name here\n project: PROJECT_NAME\n\n # add the cloud provider and plan of your choice\n # you can see all of the options at https://aiven.io/pricing\n cloudName: google-europe-west1\n plan: startup-4\n\n # general Aiven configuration\n maintenanceWindowDow: friday\n maintenanceWindowTime: 23:00:00\n userConfig:\n pg_version: \"15\"\n\n---\napiVersion: aiven.io/v1alpha1\nkind: PostgreSQL\nmetadata:\n name: read-replica-pg\nspec:\n # gets the authentication token from the `aiven-token` Secret\n authSecretRef:\n name: aiven-token\n key: token\n\n # add your project's name here\n project: PROJECT_NAME\n\n # add the cloud provider and plan of your choice\n # you can see all of the options at https://aiven.io/pricing\n cloudName: google-europe-west1\n plan: startup-4\n\n # general Aiven configuration\n maintenanceWindowDow: saturday\n maintenanceWindowTime: 23:00:00\n userConfig:\n pg_version: \"15\"\n\n # use the read_replica integration and point it to your primary service\n serviceIntegrations:\n - integrationType: read_replica\n sourceServiceName: primary-pg-service\nNote
You can create the replica service in a different region or on a different cloud provider.
2. Apply the configuration with the following command:
kubectl apply -f pg-read-replica.yaml\nThe output is similar to the following:
postgresql.aiven.io/primary-pg-service created\npostgresql.aiven.io/read-replica-pg created\n3. Check the status of the primary service with the following command:
kubectl get postgresqls.aiven.io primary-pg-service\nThe output is similar to the following:
NAME PROJECT REGION PLAN STATE\nprimary-pg-service PROJECT_NAME google-europe-west1 startup-4 RUNNING\nThe resource can be in the BUILDING state for a few minutes. After the state of the primary service changes to RUNNING, the read-only replica is created. You can check the status of the replica using the same command with the name of the replica:
kubectl get postgresqls.aiven.io read-replica-pg\nVirtual Private Cloud (VPC) peering is a method of connecting separate AWS, Google Cloud or Microsoft Azure private networks to each other. It makes it possible for the virtual machines in the different VPCs to talk to each other directly without going through the public internet.
Within the Aiven Kubernetes Operator, you can create a ProjectVPC on Aiven's side to connect to your cloud provider.
Note
Before going through this guide, make sure you have a Kubernetes cluster with the operator installed (see instructions for helm or kubectl), and a Kubernetes Secret with an Aiven authentication token.
"},{"location":"resources/project-vpc.html#create-an-aiven-vpc","title":"Create an Aiven VPC","text":"1. Create a file named vpc-sample.yaml with the following content:
apiVersion: aiven.io/v1alpha1\nkind: ProjectVPC\nmetadata:\n name: vpc-sample\nspec:\n # gets the authentication token from the `aiven-token` Secret\n authSecretRef:\n name: aiven-token\n key: token\n\n project: PROJECT_NAME\n\n # creates a VPC to link an AWS account on the South Africa region\n cloudName: aws-af-south-1\n\n # the network range used by the VPC\n networkCidr: 192.168.0.0/24\n2. Create the Project by applying the configuration:
kubectl apply -f vpc-sample.yaml\n3. Review the resource you created with the following command:
kubectl get projects.aiven.io vpc-sample\nThe output is similar to the following:
NAME PROJECT CLOUD NETWORK CIDR\nvpc-sample PROJECT_NAME aws-af-south-1 192.168.0.0/24\nFollow the official VPC documentation to complete the VPC peering on your cloud of choice.
"},{"location":"resources/project.html","title":"Aiven Project","text":"Note
Before going through this guide, make sure you have a Kubernetes cluster with the operator installed (see instructions for helm or kubectl), and a Kubernetes Secret with an Aiven authentication token.
The Project CRD allows you to create Aiven Projects, where your resources can be located.
To create a fully working Aiven Project with the Aiven Operator you need a source Aiven Project already created with a working billing configuration, like a credit card.
Create a file named project-sample.yaml with the following content:
apiVersion: aiven.io/v1alpha1\nkind: Project\nmetadata:\n name: project-sample\nspec:\n # the source Project to copy the billing information from\n copyFromProject: SOURCE_PROJECT\n\n authSecretRef:\n name: aiven-token\n key: token\n\n connInfoSecretTarget:\n name: project-sample\nApply the resource with:
kubectl apply -f project-sample.yaml\nVerify the newly created Project:
kubectl get projects.aiven.io project-sample\nThe output is similar to the following:
NAME AGE\nproject-sample 22s\nAiven for Redis\u00ae* is a fully managed in-memory NoSQL database that you can deploy in the cloud of your choice to store and access data quickly and efficiently.
Note
Before going through this guide, make sure you have a Kubernetes cluster with the operator installed (see instructions for helm or kubectl), and a Kubernetes Secret with an Aiven authentication token.
"},{"location":"resources/redis.html#create-a-redis-instance","title":"Create a Redis instance","text":"1. Create a file named redis-sample.yaml, and add the following content:
apiVersion: aiven.io/v1alpha1\nkind: Redis\nmetadata:\n name: redis-sample\nspec:\n # gets the authentication token from the `aiven-token` Secret\n authSecretRef:\n name: aiven-token\n key: token\n\n # outputs the Redis connection on the `redis-secret` Secret\n connInfoSecretTarget:\n name: redis-secret\n\n # add your Project name here\n project: PROJECT_NAME\n\n # cloud provider and plan of your choice\n # you can check all of the possibilities here https://aiven.io/pricing\n cloudName: google-europe-west1\n plan: startup-4\n\n # general Aiven configuration\n maintenanceWindowDow: friday\n maintenanceWindowTime: 23:00:00\n\n # specific Redis configuration\n userConfig:\n redis_maxmemory_policy: \"allkeys-random\"\n2. Create the service by applying the configuration:
kubectl apply -f redis-sample.yaml\n3. Review the resource you created with this command:
kubectl describe redis.aiven.io redis-sample\nThe output is similar to the following:
...\nStatus:\n Conditions:\n Last Transition Time: 2023-01-19T14:48:59Z\n Message: Successfully created or updated the instance in Aiven\n Reason: Created\n Status: True\n Type: Initialized\n Last Transition Time: 2023-01-19T14:48:59Z\n Message: Successfully created or updated the instance in Aiven, status remains unknown\n Reason: Created\n Status: Unknown\n Type: Running\n State: REBUILDING\n...\nThe resource will be in the REBUILDING state for a few minutes. Once the state changes to RUNNING, you can access the resource.
For your convenience, the operator automatically stores the Redis connection information in a Secret created with the name specified on the connInfoSecretTarget field.
To view the details of the Secret, use the following command:
kubectl describe secret redis-secret\nThe output is similar to the following:
Name: redis-secret\nNamespace: default\nLabels: <none>\nAnnotations: <none>\n\nType: Opaque\n\nData\n====\nSSL: 8 bytes\nUSER: 7 bytes\nHOST: 60 bytes\nPASSWORD: 24 bytes\nPORT: 5 bytes\nYou can use the jq to quickly decode the Secret:
kubectl get secret redis-secret -o json | jq '.data | map_values(@base64d)'\nThe output is similar to the following:
{\n \"HOST\": \"redis-sample-your-project.aivencloud.com\",\n \"PASSWORD\": \"<secret-password>\",\n \"PORT\": \"14610\",\n \"SSL\": \"required\",\n \"USER\": \"default\"\n}\nService Integrations provide additional functionality and features by connecting different Aiven services together.
See our Getting Started with Service Integrations guide for more information.
Note
Before going through this guide, make sure you have a Kubernetes cluster with the operator installed (see instructions for helm or kubectl), and a Kubernetes Secret with an Aiven authentication token.
"},{"location":"resources/service-integrations.html#send-kafka-logs-to-a-kafka-topic","title":"Send Kafka logs to a Kafka Topic","text":"This integration allows you to send Kafka service logs to a specific Kafka Topic.
First, let's create a Kafka service and a topic.
1. Create a new file named kafka-sample-topic.yaml with the following content:
apiVersion: aiven.io/v1alpha1\nkind: Kafka\nmetadata:\n name: kafka-sample\nspec:\n # gets the authentication token from the `aiven-token` Secret\n authSecretRef:\n name: aiven-token\n key: token\n\n # outputs the Kafka connection on the `kafka-connection` Secret\n connInfoSecretTarget:\n name: kafka-auth\n\n # add your Project name here\n project: PROJECT_NAME\n\n # cloud provider and plan of your choice\n # you can check all of the possibilities here https://aiven.io/pricing\n cloudName: google-europe-west1\n plan: startup-2\n\n # general Aiven configuration\n maintenanceWindowDow: friday\n maintenanceWindowTime: 23:00:00\n\n # specific Kafka configuration\n userConfig:\n kafka_version: \"2.7\"\n\n---\napiVersion: aiven.io/v1alpha1\nkind: KafkaTopic\nmetadata:\n name: logs\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n project: PROJECT_NAME\n serviceName: kafka-sample\n\n # here we can specify how many partitions the topic should have\n partitions: 3\n # and the topic replication factor\n replication: 2\n\n # we also support various topic-specific configurations\n config:\n flush_ms: 100\n2. Create the resource on Kubernetes:
kubectl apply -f kafka-sample-topic.yaml\n3. Now, create a ServiceIntegration resource to send the Kafka logs to the created topic. In the same file, add the following YAML:
apiVersion: aiven.io/v1alpha1\nkind: ServiceIntegration\nmetadata:\n name: service-integration-kafka-logs\nspec:\n # gets the authentication token from the `aiven-token` Secret\n authSecretRef:\n name: aiven-token\n key: token\n\n project: PROJECT_NAME\n\n # indicates the type of the integration\n integrationType: kafka_logs\n\n # we will send the logs to the same kafka-sample instance\n # the source and destination are the same\n sourceServiceName: kafka-sample\n destinationServiceName: kafka-sample\n\n # the topic name we will send to\n kafkaLogs:\n kafka_topic: logs\n4. Reapply the resource on Kubernetes:
kubectl apply -f kafka-sample-topic.yaml\n5. Let's check the created service integration:
kubectl get serviceintegrations.aiven.io service-integration-kafka-logs\nThe output is similar to the following:
NAME PROJECT TYPE SOURCE SERVICE NAME DESTINATION SERVICE NAME SOURCE ENDPOINT ID DESTINATION ENDPOINT ID\nservice-integration-kafka-logs your-project kafka_logs kafka-sample kafka-sample\nYour Kafka service logs are now being streamed to the logs Kafka topic.
Aiven for Apache Kafka is an excellent option if you need to run Apache Kafka at scale. With Aiven Kubernetes Operator you can get up and running with a suitably sized Apache Kafka service in a few minutes.
Note
Before going through this guide, make sure you have a Kubernetes cluster with the operator installed (see instructions for helm or kubectl), and a Kubernetes Secret with an Aiven authentication token.
"},{"location":"resources/kafka/index.html#create-a-kafka-instance","title":"Create a Kafka instance","text":"1. Create a file named kafka-sample.yaml, and add the following content:
apiVersion: aiven.io/v1alpha1\nkind: Kafka\nmetadata:\n name: kafka-sample\nspec:\n # gets the authentication token from the `aiven-token` Secret\n authSecretRef:\n name: aiven-token\n key: token\n\n # outputs the Kafka connection on the `kafka-connection` Secret\n connInfoSecretTarget:\n name: kafka-auth\n\n # add your Project name here\n project: PROJECT_NAME\n\n # cloud provider and plan of your choice\n # you can check all of the possibilities here https://aiven.io/pricing\n cloudName: google-europe-west1\n plan: startup-2\n\n # general Aiven configuration\n maintenanceWindowDow: friday\n maintenanceWindowTime: 23:00:00\n\n # specific Kafka configuration\n userConfig:\n kafka_version: \"2.7\"\n2. Create the following resource on Kubernetes:
kubectl apply -f kafka-sample.yaml\n3. Inspect the service created using the command below.
kubectl get kafka.aiven.io kafka-sample\nThe output has the project name and state, similar to the following:
NAME PROJECT REGION PLAN STATE\nkafka-sample PROJECT_NAME google-europe-west1 startup-2 RUNNING\nAfter a couple of minutes, the STATE field is changed to RUNNING, and is ready to be used.
For your convenience, the operator automatically stores the Kafka connection information in a Secret created with the name specified on the connInfoSecretTarget field.
kubectl describe secret kafka-auth\nThe output is similar to the following:
Name: kafka-auth\nNamespace: default\nAnnotations: <none>\n\nType: Opaque\n\nData\n====\nCA_CERT: 1537 bytes\nHOST: 41 bytes\nPASSWORD: 16 bytes\nPORT: 5 bytes\nUSERNAME: 8 bytes\nACCESS_CERT: 1533 bytes\nACCESS_KEY: 2484 bytes\nYou can use the jq to quickly decode the Secret:
kubectl get secret kafka-auth -o json | jq '.data | map_values(@base64d)'\nThe output is similar to the following:
{\n \"CA_CERT\": \"<secret-ca-cert>\",\n \"ACCESS_CERT\": \"<secret-cert>\",\n \"ACCESS_KEY\": \"<secret-access-key>\",\n \"HOST\": \"kafka-sample-your-project.aivencloud.com\",\n \"PASSWORD\": \"<secret-password>\",\n \"PORT\": \"13041\",\n \"USERNAME\": \"avnadmin\"\n}\nYou can verify your access to the Kafka cluster from a Pod using the authentication data from the kafka-auth Secret. kcat is used for our examples below.
1. Create a file named kafka-test-connection.yaml, and add the following content:
apiVersion: v1\nkind: Pod\nmetadata:\n name: kafka-test-connection\nspec:\n restartPolicy: Never\n containers:\n - image: edenhill/kcat:1.7.0\n name: kcat\n\n # the command below will connect to the Kafka cluster\n # and output its metadata\n command:\n [\n \"kcat\",\n \"-b\",\n \"$(HOST):$(PORT)\",\n \"-X\",\n \"security.protocol=SSL\",\n \"-X\",\n \"ssl.key.location=/kafka-auth/ACCESS_KEY\",\n \"-X\",\n \"ssl.key.password=$(PASSWORD)\",\n \"-X\",\n \"ssl.certificate.location=/kafka-auth/ACCESS_CERT\",\n \"-X\",\n \"ssl.ca.location=/kafka-auth/CA_CERT\",\n \"-L\",\n ]\n\n # loading the data from the Secret as environment variables\n # useful to access the Kafka information, like hostname and port\n envFrom:\n - secretRef:\n name: kafka-auth\n\n volumeMounts:\n - name: kafka-auth\n mountPath: \"/kafka-auth\"\n\n # loading the data from the Secret as files in a volume\n # useful to access the Kafka certificates\n volumes:\n - name: kafka-auth\n secret:\n secretName: kafka-auth\n2. Apply the file.
kubectl apply -f kafka-test-connection.yaml\nOnce successfully applied, you have a log with the metadata information about the Kafka cluster.
kubectl logs kafka-test-connection\nThe output is similar to the following:
Metadata for all topics (from broker -1: ssl://kafka-sample-your-project.aivencloud.com:13041/bootstrap):\n 3 brokers:\n broker 2 at 35.205.234.70:13041\n broker 3 at 34.77.127.70:13041 (controller)\n broker 1 at 34.78.146.156:13041\n 0 topics:\nKafkaTopic and KafkaACL","text":"To properly produce and consume content on Kafka, you need topics and ACLs. The operator supports both with the KafkaTopic and KafkaACL resources.
Below, here is how to create a Kafka topic named random-strings where random string messages will be sent.
1. Create a file named kafka-topic-random-strings.yaml with the content below:
apiVersion: aiven.io/v1alpha1\nkind: KafkaTopic\nmetadata:\n name: random-strings\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n project: PROJECT_NAME\n serviceName: kafka-sample\n\n # here we can specify how many partitions the topic should have\n partitions: 3\n # and the topic replication factor\n replication: 2\n\n # we also support various topic-specific configurations\n config:\n flush_ms: 100\n2. Create the resource on Kubernetes:
kubectl apply -f kafka-topic-random-strings.yaml\n3. Create a user and an ACL. To use the Kafka topic, create a new user with the ServiceUser resource (in order to avoid using the avnadmin superuser), and the KafkaACL to allow the user access to the topic.
In a file named kafka-acl-user-crab.yaml, add the following two resources:
apiVersion: aiven.io/v1alpha1\nkind: ServiceUser\nmetadata:\n # the name of our user \ud83e\udd80\n name: crab\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n # the Secret name we will store the users' connection information\n # looks exactly the same as the Secret generated when creating the Kafka cluster\n # we will use this Secret to produce and consume events later!\n connInfoSecretTarget:\n name: kafka-crab-connection\n\n # the Aiven project the user is related to\n project: PROJECT_NAME\n\n # the name of our Kafka Service\n serviceName: kafka-sample\n\n---\napiVersion: aiven.io/v1alpha1\nkind: KafkaACL\nmetadata:\n name: crab\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n project: PROJECT_NAME\n serviceName: kafka-sample\n\n # the username from the ServiceUser above\n username: crab\n\n # the ACL allows to produce and consume on the topic\n permission: readwrite\n\n # specify the topic we created before\n topic: random-strings\nTo create the crab user and its permissions, execute the following command:
kubectl apply -f kafka-acl-user-crab.yaml\nUsing the previously created KafkaTopic, ServiceUser, KafkaACL, you can produce and consume events.
You can use kcat to produce a message into Kafka, and the -t random-strings argument to select the desired topic, and use the content of the /etc/issue file as the message's body.
1. Create a kafka-crab-produce.yaml file with the content below:
apiVersion: v1\nkind: Pod\nmetadata:\n name: kafka-crab-produce\nspec:\n restartPolicy: Never\n containers:\n - image: edenhill/kcat:1.7.0\n name: kcat\n\n # the command below will produce a message with the /etc/issue file content\n command:\n [\n \"kcat\",\n \"-b\",\n \"$(HOST):$(PORT)\",\n \"-X\",\n \"security.protocol=SSL\",\n \"-X\",\n \"ssl.key.location=/crab-auth/ACCESS_KEY\",\n \"-X\",\n \"ssl.key.password=$(PASSWORD)\",\n \"-X\",\n \"ssl.certificate.location=/crab-auth/ACCESS_CERT\",\n \"-X\",\n \"ssl.ca.location=/crab-auth/CA_CERT\",\n \"-P\",\n \"-t\",\n \"random-strings\",\n \"/etc/issue\",\n ]\n\n # loading the crab user data from the Secret as environment variables\n # useful to access the Kafka information, like hostname and port\n envFrom:\n - secretRef:\n name: kafka-crab-connection\n\n volumeMounts:\n - name: crab-auth\n mountPath: \"/crab-auth\"\n\n # loading the crab user information from the Secret as files in a volume\n # useful to access the Kafka certificates\n volumes:\n - name: crab-auth\n secret:\n secretName: kafka-crab-connection\n2. Create the Pod with the following content:
kubectl apply -f kafka-crab-produce.yaml\nNow your event is stored in Kafka.
To consume a message, you can use a graphical interface called Kowl. It allows you to explore information about our Kafka cluster, such as brokers, topics, or consumer groups.
1. Create a Kubernetes Pod and service to deploy and access Kowl. Create a file named kafka-crab-consume.yaml with the content below:
apiVersion: v1\nkind: Pod\nmetadata:\n name: kafka-crab-consume\n labels:\n app: kafka-crab-consume\nspec:\n containers:\n - image: quay.io/cloudhut/kowl:v1.4.0\n name: kowl\n\n # kowl configuration values\n env:\n - name: KAFKA_TLS_ENABLED\n value: \"true\"\n\n - name: KAFKA_BROKERS\n value: $(HOST):$(PORT)\n - name: KAFKA_TLS_PASSPHRASE\n value: $(PASSWORD)\n\n - name: KAFKA_TLS_CAFILEPATH\n value: /crab-auth/CA_CERT\n - name: KAFKA_TLS_CERTFILEPATH\n value: /crab-auth/ACCESS_CERT\n - name: KAFKA_TLS_KEYFILEPATH\n value: /crab-auth/ACCESS_KEY\n\n # inject all connection information as environment variables\n envFrom:\n - secretRef:\n name: kafka-crab-connection\n\n volumeMounts:\n - name: crab-auth\n mountPath: /crab-auth\n\n # loading the crab user information from the Secret as files in a volume\n # useful to access the Kafka certificates\n volumes:\n - name: crab-auth\n secret:\n secretName: kafka-crab-connection\n\n---\n# we will be using a simple service to access Kowl on port 8080\napiVersion: v1\nkind: Service\nmetadata:\n name: kafka-crab-consume\nspec:\n selector:\n app: kafka-crab-consume\n ports:\n - port: 8080\n targetPort: 8080\n2. Create the resources with:
kubectl apply -f kafka-crab-consume.yaml\n3. In another terminal create a port-forward tunnel to your Pod:
kubectl port-forward kafka-crab-consume 8080:8080\n4. In the browser of your choice, access the http://localhost:8080 address. You now see a page with the random-strings topic listed:
5. Click the topic name to see the message.
You have now consumed the message.
"},{"location":"resources/kafka/connect.html","title":"Kafka Connect","text":"Aiven for Apache Kafka Connect is a framework and a runtime for integrating Kafka with other systems. Kafka connectors can either be a source (for pulling data from other systems into Kafka) or sink (for pushing data into other systems from Kafka).
This section involves a few different Kubernetes CRDs:
KafkaService service with a KafkaTopicKafkaConnect serviceServiceIntegration to integrate the Kafka and KafkaConnect servicesPostgreSQL used as a sink to receive messages from KafkaKafkaConnector to finally connect the Kafka with the PostgreSQLCreate a file named kafka-sample-connect.yaml with the following content:
apiVersion: aiven.io/v1alpha1\nkind: Kafka\nmetadata:\n name: kafka-sample-connect\nspec:\n # gets the authentication token from the `aiven-token` Secret\n authSecretRef:\n name: aiven-token\n key: token\n\n # outputs the Kafka connection on the `kafka-connection` Secret\n connInfoSecretTarget:\n name: kafka-auth\n\n # add your Project name here\n project: PROJECT_NAME\n\n # cloud provider and plan of your choice\n # you can check all of the possibilities here https://aiven.io/pricing\n cloudName: google-europe-west1\n plan: business-4\n\n # general Aiven configuration\n maintenanceWindowDow: friday\n maintenanceWindowTime: 23:00:00\n\n # specific Kafka configuration\n userConfig:\n kafka_version: \"2.7\"\n kafka_connect: true\n\n---\napiVersion: aiven.io/v1alpha1\nkind: KafkaTopic\nmetadata:\n name: kafka-topic-connect\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n project: PROJECT_NAME\n serviceName: kafka-sample-connect\n\n replication: 2\n partitions: 1\nNext, create a file named kafka-connect.yaml and add the following KafkaConnect resource:
apiVersion: aiven.io/v1alpha1\nkind: KafkaConnect\nmetadata:\n name: kafka-connect\nspec:\n # gets the authentication token from the `aiven-token` Secret\n authSecretRef:\n name: aiven-token\n key: token\n\n # add your Project name here\n project: PROJECT_NAME\n\n # cloud provider and plan of your choice\n # you can check all of the possibilities here https://aiven.io/pricing\n cloudName: google-europe-west1\n plan: startup-4\n\n # general Aiven configuration\n maintenanceWindowDow: friday\n maintenanceWindowTime: 23:00:00\nNow let's create a ServiceIntegration. It will use the fields sourceServiceName and destinationServiceName to integrate the previously created kafka-sample-connect and kafka-connect. Open a new file named service-integration-connect.yaml and add the content below:
apiVersion: aiven.io/v1alpha1\nkind: ServiceIntegration\nmetadata:\n name: service-integration-kafka-connect\nspec:\n # gets the authentication token from the `aiven-token` Secret\n authSecretRef:\n name: aiven-token\n key: token\n\n project: PROJECT_NAME\n\n # indicates the type of the integration\n integrationType: kafka_connect\n\n # we will send messages from the `kafka-sample-connect` to `kafka-connect`\n sourceServiceName: kafka-sample-connect\n destinationServiceName: kafka-connect\nLet's add an Aiven for PostgreSQL service. It will be the service used as a sink, receiving messages from the kafka-sample-connect cluster. Create a file named pg-sample-connect.yaml with the content below:
apiVersion: aiven.io/v1alpha1\nkind: PostgreSQL\nmetadata:\n name: pg-connect\nspec:\n # gets the authentication token from the `aiven-token` Secret\n authSecretRef:\n name: aiven-token\n key: token\n\n # outputs the PostgreSQL connection on the `pg-connection` Secret\n connInfoSecretTarget:\n name: pg-connection\n\n # add your Project name here\n project: PROJECT_NAME\n\n # cloud provider and plan of your choice\n # you can check all of the possibilities here https://aiven.io/pricing\n cloudName: google-europe-west1\n plan: startup-4\n\n # general Aiven configuration\n maintenanceWindowDow: friday\n maintenanceWindowTime: 23:00:00\nFinally, let's add the glue of everything: a KafkaConnector. As described in the specification, it will send receive messages from the kafka-sample-connect and send them to the pg-connect service. Check our official documentation for more connectors.
Create a file named kafka-connector-connect.yaml and with the content below:
apiVersion: aiven.io/v1alpha1\nkind: KafkaConnector\nmetadata:\n name: kafka-connector\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n project: PROJECT_NAME\n\n # the Kafka cluster name\n serviceName: kafka-sample-connect\n\n # the connector we will be using\n connectorClass: io.aiven.connect.jdbc.JdbcSinkConnector\n\n userConfig:\n auto.create: \"true\"\n\n # constructs the pg-connect connection information\n connection.url: 'jdbc:postgresql://{{ fromSecret \"pg-connection\" \"PGHOST\"}}:{{ fromSecret \"pg-connection\" \"PGPORT\" }}/{{ fromSecret \"pg-connection\" \"PGDATABASE\" }}'\n connection.user: '{{ fromSecret \"pg-connection\" \"PGUSER\" }}'\n connection.password: '{{ fromSecret \"pg-connection\" \"PGPASSWORD\" }}'\n\n # specify which topics it will watch\n topics: kafka-topic-connect\n\n key.converter: org.apache.kafka.connect.json.JsonConverter\n value.converter: org.apache.kafka.connect.json.JsonConverter\n value.converter.schemas.enable: \"true\"\nWith all the files created, apply the new Kubernetes resources:
kubectl apply \\\n -f kafka-sample-connect.yaml \\\n -f kafka-connect.yaml \\\n -f service-integration-connect.yaml \\\n -f pg-sample-connect.yaml \\\n -f kafka-connector-connect.yaml\nIt will take some time for all the services to be up and running. You can check their status with the following command:
kubectl get \\\n kafkas.aiven.io/kafka-sample-connect \\\n kafkaconnects.aiven.io/kafka-connect \\\n postgresqls.aiven.io/pg-connect \\\n kafkaconnectors.aiven.io/kafka-connector\nThe output is similar to the following:
NAME PROJECT REGION PLAN STATE\nkafka.aiven.io/kafka-sample-connect your-project google-europe-west1 business-4 RUNNING\n\nNAME STATE\nkafkaconnect.aiven.io/kafka-connect RUNNING\n\nNAME PROJECT REGION PLAN STATE\npostgresql.aiven.io/pg-connect your-project google-europe-west1 startup-4 RUNNING\n\nNAME SERVICE NAME PROJECT CONNECTOR CLASS STATE TASKS TOTAL TASKS RUNNING\nkafkaconnector.aiven.io/kafka-connector kafka-sample-connect your-project io.aiven.connect.jdbc.JdbcSinkConnector RUNNING 1 1\nThe deployment is finished when all services have the state RUNNING.
To test the connection integration, let's produce a Kafka message using kcat from within the Kubernetes cluster. We will deploy a Pod responsible for crafting a message and sending to the Kafka cluster, using the kafka-auth secret generate by the Kafka CRD.
Create a new file named kcat-connect.yaml and add the content below:
apiVersion: v1\nkind: Pod\nmetadata:\n name: kafka-message\nspec:\n containers:\n\n restartPolicy: Never\n - image: edenhill/kcat:1.7.0\n name: kcat\n\n command: ['/bin/sh']\n args: [\n '-c',\n 'echo {\\\"schema\\\":{\\\"type\\\":\\\"struct\\\",\\\"fields\\\":[{ \\\"field\\\": \\\"text\\\", \\\"type\\\": \\\"string\\\", \\\"optional\\\": false } ] }, \\\"payload\\\": { \\\"text\\\": \\\"Hello World\\\" } } > /tmp/msg;\n\n kcat\n -b $(HOST):$(PORT)\n -X security.protocol=SSL\n -X ssl.key.location=/kafka-auth/ACCESS_KEY\n -X ssl.key.password=$(PASSWORD)\n -X ssl.certificate.location=/kafka-auth/ACCESS_CERT\n -X ssl.ca.location=/kafka-auth/CA_CERT\n -P -t kafka-topic-connect /tmp/msg'\n ]\n\n envFrom:\n - secretRef:\n name: kafka-auth\n\n volumeMounts:\n - name: kafka-auth\n mountPath: \"/kafka-auth\"\n\n volumes:\n - name: kafka-auth\n secret:\n secretName: kafka-auth\nApply the file with:
kubectl apply -f kcat-connect.yaml\nThe Pod will execute the commands and finish. You can confirm its Completed state with:
kubectl get pod kafka-message\nThe output is similar to the following:
NAME READY STATUS RESTARTS AGE\nkafka-message 0/1 Completed 0 5m35s\nIf everything went smoothly, we should have our produced message in the PostgreSQL service. Let's check that out.
Create a file named psql-connect.yaml with the content below:
apiVersion: v1\nkind: Pod\nmetadata:\n name: psql-connect\nspec:\n restartPolicy: Never\n containers:\n - image: postgres:13\n name: postgres\n # \"kafka-topic-connect\" is the table automatically created by KafkaConnect\n command:\n [\n \"psql\",\n \"$(DATABASE_URI)\",\n \"-c\",\n 'SELECT * from \"kafka-topic-connect\";',\n ]\n\n envFrom:\n - secretRef:\n name: pg-connection\nApply the file with:
kubectl apply -f psql-connect.yaml\nAfter a couple of seconds, inspect its log with this command:
kubectl logs psql-connect\nThe output is similar to the following:
text\n-------------\n Hello World\n(1 row)\nTo clean up all the created resources, use the following command:
kubectl delete \\\n -f kafka-sample-connect.yaml \\\n -f kafka-connect.yaml \\\n -f service-integration-connect.yaml \\\n -f pg-sample-connect.yaml \\\n -f kafka-connector-connect.yaml \\\n -f kcat-connect.yaml \\\n -f psql-connect.yaml\nKafkaSchema","text":"Aiven develops and maintain Karapace, an open source implementation of Kafka REST and schema registry. Is available out of the box for our managed Kafka service.
The schema registry address and authentication is the same as the Kafka broker, the only different is the usage of the port 13044.
First, let's create an Aiven for Apache Kafka service.
1. Create a file named kafka-sample-schema.yaml and add the content below:
apiVersion: aiven.io/v1alpha1\nkind: Kafka\nmetadata:\n name: kafka-sample-schema\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n connInfoSecretTarget:\n name: kafka-auth\n\n project: PROJECT_NAME\n cloudName: google-europe-west1\n plan: startup-2\n maintenanceWindowDow: friday\n maintenanceWindowTime: 23:00:00\n\n userConfig:\n kafka_version: \"2.7\"\n\n # this flag enables the Schema registry\n schema_registry: true\n2. Apply the changes with the following command:
kubectl apply -f kafka-schema.yaml\nNow, let's create the schema itself.
1. Create a new file named kafka-sample-schema.yaml and add the YAML content below:
apiVersion: aiven.io/v1alpha1\nkind: KafkaSchema\nmetadata:\n name: kafka-schema\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n project: PROJECT_NAME\n serviceName: kafka-sample-schema\n\n # the name of the Schema\n subjectName: MySchema\n\n # the schema itself, in JSON format\n schema: |\n {\n \"type\": \"record\",\n \"name\": \"MySchema\",\n \"fields\": [\n {\n \"name\": \"field\",\n \"type\": \"string\"\n }\n ]\n }\n\n # sets the schema compatibility level\n compatibilityLevel: BACKWARD\n2. Create the schema with the command:
kubectl apply -f kafka-schema.yaml\n3. Review the resource you created with the following command:
kubectl get kafkaschemas.aiven.io kafka-schema\nThe output is similar to the following:
NAME SERVICE NAME PROJECT SUBJECT COMPATIBILITY LEVEL VERSION\nkafka-schema kafka-sample PROJECT_NAME MySchema BACKWARD 1\nNow you can follow the instructions to use a schema registry in Java on how to use the schema created.
"}]} \ No newline at end of file +var __index = {"config":{"lang":["en"],"separator":"[\\s\\-]+","pipeline":["stopWordFilter"]},"docs":[{"location":"index.html","title":"Welcome to Aiven Operator for Kubernetes","text":"Provision and manage Aiven services from your Kubernetes cluster.
"},{"location":"index.html#what-is-aiven","title":"What is Aiven?","text":"Aiven offers managed services for the best open source data technologies, on a cloud of your choice.
We offer multiple cloud options because we believe that everyone should have access to great data platforms wherever they host their applications. Our customers tell us they love it because they know that they aren\u2019t locked in to one particular cloud platform for all their data needs.
"},{"location":"index.html#contributing","title":"Contributing","text":"The contribution guide covers everything you need to know about how you can contribute to Aiven Operator for Kubernetes. The developer guide will help you onboard as a developer.
"},{"location":"authentication.html","title":"Authenticating","text":"To get authenticated and authorized, set up the communication between the Aiven Operator for Kubernetes and Aiven by using a token stored in a Kubernetes Secret. You can then refer to the Secret name on every custom resource in the authSecretRef field.
Important
If you don't have an Aiven account yet, sign up here for a free trial \ud83e\udd80
1. Generate an authentication token
Refer to our documentation article to generate your authentication token.
2. Create the Kubernetes Secret
The following command creates a Secret named aiven-token with a token field containing the authentication token:
kubectl create secret generic aiven-token --from-literal=token=\"TOKEN\"\nWhen managing your Aiven resources, we will be using the created Secret in the authSecretRef field. It will look like the example below:
apiVersion: aiven.io/v1alpha1\nkind: PostgreSQL\nmetadata:\n name: pg-sample\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n [ ... ]\nAlso, note that within Aiven, all resources are conceptually inside a Project. By default, a random project name is generated when you signup, but you can also create new projects.
The Project name is required in most of the resources. It will look like the example below:
apiVersion: aiven.io/v1alpha1\nkind: PostgreSQL\nmetadata:\n name: pg-sample\nspec:\n project: PROJECT_NAME\n [ ... ]\nPostgreSQL: wait for a valid backup to create read replicaClickhouseGrant: grant privileges for an unknown table (Clickhouse can do that)ClickhouseGrant: track the state to revoke only known privilegesCassandra field userConfig.cassandra.read_request_timeout_in_ms, type integer: How long the coordinator waits for read operations to complete before timing it outCassandra field userConfig.cassandra.write_request_timeout_in_ms, type integer: How long the coordinator waits for write requests to complete with at least one node in the local datacenterOpenSearch field userConfig.opensearch.knn_memory_circuit_breaker_enabled, type boolean: Enable or disable KNN memory circuit breaker. Defaults to trueOpenSearch field userConfig.opensearch.knn_memory_circuit_breaker_limit, type integer: Maximum amount of memory that can be used for KNN index. Defaults to 50% of the JVM heap sizePostgreSQL field userConfig.pg.log_line_prefix: enum ['%m [%p] %q[user=%u,db=%d,app=%a] ', '%t [%p]: [%l-1] user=%u,db=%d,app=%a,client=%h ', 'pid=%p,user=%u,db=%d,app=%a,client=%h '] \u2192 ['%m [%p] %q[user=%u,db=%d,app=%a] ', '%t [%p]: [%l-1] user=%u,db=%d,app=%a,client=%h ', 'pid=%p,user=%u,db=%d,app=%a,client=%h ', 'pid=%p,user=%u,db=%d,app=%a,client=%h,txid=%x,qid=%Q ']http.StatusBadRequest on ClickhouseGrant deletionClickhouseGrant invalid remote and local privileges comparisonClickhouseGrant: doesn't escape role name to grantClickhouseUser: password was reset due to an incorrect processing cycleClickhouseRole deletion error (missing database)ClickhouseGrant deletion errors (missing database, service, role)REBALANCING stateClickhouseGrantKafkaConnect field userConfig.secret_providers, type array: Configure external secret providers in order to reference external secrets in connector configurationKafka field userConfig.kafka_connect_secret_providers, type array: Configure external secret providers in order to reference external secrets in connector configurationKafka field userConfig.letsencrypt_sasl_privatelink, type boolean: Use Letsencrypt CA for Kafka SASL via PrivatelinkServiceIntegration field datadog.mirrormaker_custom_metrics, type array: List of custom metricsServiceIntegration field kafkaMirrormaker.kafka_mirrormaker.consumer_auto_offset_reset, type string: Set where consumer starts to consume dataServiceIntegration field kafkaMirrormaker.kafka_mirrormaker.consumer_max_poll_records, type integer: Set consumer max.poll.records. The default is 500PostgreSQL field userConfig.pgaudit: deprecatedServiceIntegrationEndpoint field externalPostgresql.ssl_mode: enum [allow, disable, prefer, require, verify-ca, verify-full] \u2192 [require, verify-ca, verify-full]ServiceIntegrationEndpointServiceIntegration flink_external_postgresql typeServiceIntegration field datadog.datadog_pgbouncer_enabled, type boolean: Enable Datadog PgBouncer Metric TrackingServiceIntegration deletion when instance has no id setdisk_space field validationproject, serviceName fields validationConnectionPool doesn't check service user preconditionCA_CERT secret key for Grafana, OpenSearch, Redis, and Clickhouse. Can't be used with these service types ddog-gov.com, us3.datadoghq.com, us5.datadoghq.com]`ServiceIntegrationEndpoint field externalKafka.ssl_endpoint_identification_algorithm: enum [, https] \u2192 [https]ClickhouseUser webhook. Doesn't do any validation or mutationKafka field userConfig.kafka_version: enum [3.4, 3.5, 3.6] \u2192 [3.4, 3.5, 3.6, 3.7]ServiceIntegrationEndpoint field datadog.site: enum [datadoghq.com, datadoghq.eu, ddog-gov.com, us3.datadoghq.com, us5.datadoghq.com] \u2192 `[ap1.datadoghq.com, datadoghq.com, datadoghq.eu,ClickhouseRoleKafkaACLKafkaSchemaRegistryACL kindClickhouseDatabase kindDatabase kind validations and default values with CRD validation rulesREDIS_CA_CERT, MYSQL_CA_CERT, etc.KafkaTopic field config.local_retention_bytes, type integer: local.retention.bytes valueKafkaTopic field config.local_retention_ms, type integer: local.retention.ms valueKafkaTopic field config.remote_storage_enable, type boolean: remote_storage_enableCassandra field userConfig.cassandra_version: pattern ^[0-9]+(\\.[0-9]+)?$Cassandra field userConfig.project_to_fork_from: pattern ^[a-z][-a-z0-9]{0,63}$|^$Cassandra field userConfig.service_to_fork_from: pattern ^[a-z][-a-z0-9]{0,63}$|^$Cassandra field userConfig.service_to_join_with: pattern ^[a-z][-a-z0-9]{0,63}$Clickhouse field userConfig.project_to_fork_from: pattern ^[a-z][-a-z0-9]{0,63}$|^$Clickhouse field userConfig.service_to_fork_from: pattern ^[a-z][-a-z0-9]{0,63}$|^$Grafana field userConfig.project_to_fork_from: pattern ^[a-z][-a-z0-9]{0,63}$|^$Grafana field userConfig.service_to_fork_from: pattern ^[a-z][-a-z0-9]{0,63}$|^$Kafka field userConfig.kafka.sasl_oauthbearer_expected_audience: pattern ^[^\\r\\n]*$Kafka field userConfig.kafka.sasl_oauthbearer_expected_issuer: pattern ^[^\\r\\n]*$Kafka field userConfig.kafka.sasl_oauthbearer_sub_claim_name: pattern ^[^\\r\\n]*$MySQL field userConfig.mysql.default_time_zone: pattern ^([-+][\\d:]*|[\\w/]*)$MySQL field userConfig.project_to_fork_from: pattern ^[a-z][-a-z0-9]{0,63}$|^$MySQL field userConfig.service_to_fork_from: pattern ^[a-z][-a-z0-9]{0,63}$|^$OpenSearch field userConfig.openid.client_id: pattern ^[^\\r\\n]*$OpenSearch field userConfig.openid.client_secret: pattern ^[^\\r\\n]*$OpenSearch field userConfig.openid.header: pattern ^[^\\r\\n]*$OpenSearch field userConfig.openid.jwt_header: pattern ^[^\\r\\n]*$OpenSearch field userConfig.openid.jwt_url_parameter: pattern ^[^\\r\\n]*$OpenSearch field userConfig.openid.roles_key: pattern ^[^\\r\\n]*$OpenSearch field userConfig.openid.scope: pattern ^[^\\r\\n]*$OpenSearch field userConfig.openid.subject_key: pattern ^[^\\r\\n]*$OpenSearch field userConfig.project_to_fork_from: pattern ^[a-z][-a-z0-9]{0,63}$|^$OpenSearch field userConfig.saml.idp_entity_id: pattern ^[^\\r\\n]*$OpenSearch field userConfig.saml.roles_key: pattern ^[^\\r\\n]*$OpenSearch field userConfig.saml.sp_entity_id: pattern ^[^\\r\\n]*$OpenSearch field userConfig.saml.subject_key: pattern ^[^\\r\\n]*$OpenSearch field userConfig.service_to_fork_from: pattern ^[a-z][-a-z0-9]{0,63}$|^$PostgreSQL field userConfig.pg.timezone: pattern ^[\\w/]*$PostgreSQL field userConfig.pg_service_to_fork_from: pattern ^[a-z][-a-z0-9]{0,63}$|^$PostgreSQL field userConfig.project_to_fork_from: pattern ^[a-z][-a-z0-9]{0,63}$|^$PostgreSQL field userConfig.service_to_fork_from: pattern ^[a-z][-a-z0-9]{0,63}$|^$Redis field userConfig.project_to_fork_from: pattern ^[a-z][-a-z0-9]{0,63}$|^$Redis field userConfig.service_to_fork_from: pattern ^[a-z][-a-z0-9]{0,63}$|^$OpenSearch field userConfig.opensearch.plugins_alerting_filter_by_backend_roles, type boolean: Enable or disable filtering of alerting by backend roles. Requires Security pluginRedis field userConfig.redis_notify_keyspace_events: pattern ^[KEg\\$lshzxeA]*$ \u2192 ^[KEg\\$lshzxentdmA]*$PostgreSQL field userConfig.pgaudit, type object: System-wide settings for the pgaudit extensionServiceIntegration field datadog.opensearch.cluster_stats_enabled, type boolean: Enable Datadog Opensearch Cluster MonitoringOpenSearch field userConfig.opensearch.enable_security_audit, type boolean: Enable/Disable security auditKafka field userConfig.kafka_rest_config.name_strategy, type string: Name strategy to use when selecting subject for storing schemasRedis field userConfig.redis_version, type string: Redis major versionGrafana field userConfig.auth_github.auto_login, type boolean: Allow users to bypass the login screen and automatically log inGrafana field userConfig.auth_github.skip_org_role_sync, type boolean: Stop automatically syncing user rolesClickhouse field userConfig.additional_backup_regions: deprecatedGrafana field userConfig.additional_backup_regions: deprecatedKafkaConnect field userConfig.additional_backup_regions: deprecatedKafka field userConfig.additional_backup_regions: deprecatedOpenSearch field userConfig.additional_backup_regions: deprecatedRedis field userConfig.additional_backup_regions: deprecatedCassandra field userConfig.cassandra_version: enum [3, 4, 4.1] \u2192 [4, 4.1]Kafka field userConfig.kafka_version: enum [3.1, 3.3, 3.4, 3.5, 3.6] \u2192 [3.4, 3.5, 3.6]PostgreSQL field userConfig.pg_version: enum [11, 12, 13, 14, 15, 16] \u2192 [12, 13, 14, 15, 16]Cassandra field technicalEmails, type array: Defines the email addresses that will receive alerts about upcoming maintenance updates or warnings about service instabilityClickhouse field technicalEmails, type array: Defines the email addresses that will receive alerts about upcoming maintenance updates or warnings about service instabilityGrafana field technicalEmails, type array: Defines the email addresses that will receive alerts about upcoming maintenance updates or warnings about service instabilityKafkaConnect field technicalEmails, type array: Defines the email addresses that will receive alerts about upcoming maintenance updates or warnings about service instabilityKafka field technicalEmails, type array: Defines the email addresses that will receive alerts about upcoming maintenance updates or warnings about service instabilityMySQL field technicalEmails, type array: Defines the email addresses that will receive alerts about upcoming maintenance updates or warnings about service instabilityOpenSearch field technicalEmails, type array: Defines the email addresses that will receive alerts about upcoming maintenance updates or warnings about service instabilityPostgreSQL field technicalEmails, type array: Defines the email addresses that will receive alerts about upcoming maintenance updates or warnings about service instabilityRedis field technicalEmails, type array: Defines the email addresses that will receive alerts about upcoming maintenance updates or warnings about service instabilityCassandra field connInfoSecretTargetDisabled, type boolean: When true, the secret containing connection information will not be created, defaults to falseClickhouse field connInfoSecretTargetDisabled, type boolean: When true, the secret containing connection information will not be created, defaults to falseClickhouseUser field connInfoSecretTargetDisabled, type boolean: When true, the secret containing connection information will not be created, defaults to falseConnectionPool field connInfoSecretTargetDisabled, type boolean: When true, the secret containing connection information will not be created, defaults to falseGrafana field connInfoSecretTargetDisabled, type boolean: When true, the secret containing connection information will not be created, defaults to falseKafka field connInfoSecretTargetDisabled, type boolean: When true, the secret containing connection information will not be created, defaults to falseMySQL field connInfoSecretTargetDisabled, type boolean: When true, the secret containing connection information will not be created, defaults to falseOpenSearch field connInfoSecretTargetDisabled, type boolean: When true, the secret containing connection information will not be created, defaults to falsePostgreSQL field connInfoSecretTargetDisabled, type boolean: When true, the secret containing connection information will not be created, defaults to falseProject field connInfoSecretTargetDisabled, type boolean: When true, the secret containing connection information will not be created, defaults to falseRedis field connInfoSecretTargetDisabled, type boolean: When true, the secret containing connection information will not be created, defaults to falseServiceUser field connInfoSecretTargetDisabled, type boolean: When true, the secret containing connection information will not be created, defaults to falseKAFKA_SCHEMA_REGISTRY_URI and KAFKA_REST_URI to Kafka secretCONNECTIONPOOL_NAME in ConnectionPool secretCONNECTIONPOOL_PORT exposes service port instead of pool portSERVICEUSER_PORT when sasl is the only authentication methodPostgreSQL field userConfig.pg_qualstats.enabled: deprecatedPostgreSQL field userConfig.pg_qualstats.min_err_estimate_num: deprecatedPostgreSQL field userConfig.pg_qualstats.min_err_estimate_ratio: deprecatedPostgreSQL field userConfig.pg_qualstats.track_constants: deprecatedPostgreSQL field userConfig.pg_qualstats.track_pg_catalog: deprecatedPreconditions, CreateOrUpdate, Delete. Thanks to @ataraxKafka field userConfig.kafka.transaction_partition_verification_enable, type boolean: Enable verification that checks that the partition has been added to the transaction before writing transactional records to the partitionCassandra field userConfig.service_log, type boolean: Store logs for the service so that they are available in the HTTP API and consoleClickhouse field userConfig.service_log, type boolean: Store logs for the service so that they are available in the HTTP API and consoleGrafana field userConfig.service_log, type boolean: Store logs for the service so that they are available in the HTTP API and consoleKafkaConnect field userConfig.service_log, type boolean: Store logs for the service so that they are available in the HTTP API and consoleKafka field userConfig.kafka_rest_config.name_strategy_validation, type boolean: If true, validate that given schema is registered under expected subject name by the used name strategy when producing messagesKafka field userConfig.service_log, type boolean: Store logs for the service so that they are available in the HTTP API and consoleMySQL field userConfig.service_log, type boolean: Store logs for the service so that they are available in the HTTP API and consoleOpenSearch field userConfig.service_log, type boolean: Store logs for the service so that they are available in the HTTP API and consolePostgreSQL field userConfig.pg_qualstats, type object: System-wide settings for the pg_qualstats extensionPostgreSQL field userConfig.service_log, type boolean: Store logs for the service so that they are available in the HTTP API and consoleRedis field userConfig.service_log, type boolean: Store logs for the service so that they are available in the HTTP API and consoleServiceIntegration: do not send empty user config to the APIstring type fields to the documentationClickhouse field userConfig.private_access.clickhouse_mysql, type boolean: Allow clients to connect to clickhouse_mysql with a DNS name that always resolves to the service's private IP addressesClickhouse field userConfig.privatelink_access.clickhouse_mysql, type boolean: Enable clickhouse_mysqlClickhouse field userConfig.public_access.clickhouse_mysql, type boolean: Allow clients to connect to clickhouse_mysql from the public internet for service nodes that are in a project VPC or another type of private networkGrafana field userConfig.unified_alerting_enabled, type boolean: Enable or disable Grafana unified alerting functionalityKafka field userConfig.aiven_kafka_topic_messages, type boolean: Allow access to read Kafka topic messages in the Aiven Console and REST APIKafka field userConfig.kafka.sasl_oauthbearer_expected_audience, type string: The (optional) comma-delimited setting for the broker to use to verify that the JWT was issued for one of the expected audiencesKafka field userConfig.kafka.sasl_oauthbearer_expected_issuer, type string: Optional setting for the broker to use to verify that the JWT was created by the expected issuerKafka field userConfig.kafka.sasl_oauthbearer_jwks_endpoint_url, type string: OIDC JWKS endpoint URL. By setting this the SASL SSL OAuth2/OIDC authentication is enabledKafka field userConfig.kafka.sasl_oauthbearer_sub_claim_name, type string: Name of the scope from which to extract the subject claim from the JWT. Defaults to subKafka field userConfig.kafka_version: enum [3.1, 3.3, 3.4, 3.5] \u2192 [3.1, 3.3, 3.4, 3.5, 3.6]Kafka field userConfig.tiered_storage.local_cache.size: deprecatedOpenSearch field userConfig.opensearch.indices_memory_max_index_buffer_size, type integer: Absolute value. Default is unbound. Doesn't work without indices.memory.index_buffer_sizeOpenSearch field userConfig.opensearch.indices_memory_min_index_buffer_size, type integer: Absolute value. Default is 48mb. Doesn't work without indices.memory.index_buffer_sizeOpenSearch field userConfig.opensearch.auth_failure_listeners.internal_authentication_backend_limiting.authentication_backend: enum [internal]OpenSearch field userConfig.opensearch.auth_failure_listeners.internal_authentication_backend_limiting.type: enum [username]OpenSearch field userConfig.opensearch.auth_failure_listeners.ip_rate_limiting.type: enum [ip]OpenSearch field userConfig.opensearch.search_max_buckets: maximum 65536 \u2192 1000000ServiceIntegration field kafkaMirrormaker.kafka_mirrormaker.producer_max_request_size: maximum 67108864 \u2192 268435456projectVpcId and projectVPCRef mutablenil user config conversionCassandra kind option additional_backup_regionsGrafana kind option auto_loginKafka kind properties log_local_retention_bytes, log_local_retention_msKafka kind option remote_log_storage_system_enableOpenSearch kind option auth_failure_listenersOpenSearch kind Index State Management optionsKafkaKafka version 3.5Kafka spec property scheduled_rebalance_max_delay_msKafka spec property remote_log_storage_system_enableKafkaConnect spec property scheduled_rebalance_max_delay_msOpenSearch spec property openidKAFKA_SCHEMA_REGISTRY_HOST and KAFKA_SCHEMA_REGISTRY_PORT for KafkaKAFKA_CONNECT_HOST, KAFKA_CONNECT_PORT, KAFKA_REST_HOST and KAFKA_REST_PORT for Kafka. Thanks to @Dariuschunclean_leader_election_enable from KafkaTopic kind configKAFKA_SASL_PORT for Kafka kind if SASL authentication method is enabledredis options to datadog ServiceIntegrationCassandra version 3Kafka versions 3.1 and 3.4kafka_rest_config.producer_max_request_size optionkafka_mirrormaker.producer_compression_type optionclusterRole.create option to Helm chart. Thanks to @ryaneorthOpenSearch.spec.userConfig.idp_pemtrustedcas_content option. Specifies the PEM-encoded root certificate authority (CA) content for the SAML identity provider (IdP) server verification.ServiceIntegration kind SourceProjectName and DestinationProjectName fieldsServiceIntegration fields MaxLength validationServiceIntegration validation: multiple user configs cannot be setServiceIntegration, should not require destinationServiceName or sourceEndpointID fieldServiceIntegration, add missing external_aws_cloudwatch_metrics type config serializationServiceIntegration integration type listannotations and labels fields to connInfoSecretTargetOpenSearch.spec.userConfig.opensearch.search_max_buckets maximum to 65536plan as a required fieldminumim, maximum validations for number typeip_filter backward compatibilityclickhouseKafka.tables.data_format-property enum RawBLOB valueuserConfig.opensearch.email_sender_username validation patternlog_cleaner_min_cleanable_ratio minimum and maximum validation rules3.2, reached EOL10, reached EOLProjectVPC by ID to avoid conflictsProjectVPC deletion by exiting on DELETING statusClickhouseUser controllerClickhouseUser.spec.project and ClickhouseUser.spec.serviceName as immutablesignalfxAuthSecretRef fields marked as requireddatadog, kafka_connect, kafka_logs, metricsclickhouse_postgresql, clickhouse_kafka, clickhouse_kafka, logs, external_aws_cloudwatch_metricsKafkaTopic.Spec.topicName field. Unlike the metadata.name, supports additional characters and has a longer length. KafkaTopic.Spec.topicName replaces metadata.name in future releases and will be marked as required.false value for termination_protection propertymin_cleanable_dirty_ratio. Thanks to @TV2rdImportant: This release brings breaking changes to the userConfig property. After new charts are installed, update your existing instances manually using the kubectl edit command according to the API reference.
Note: It is now recommended to disable webhooks for Kubernetes version 1.25 and higher, as native CRD validation rules are used.
ip_filter field is now of object typeserviceIntegrations on service types. Only the read_replica type is available.min_cleanable_dirty_ratio config field supportspec.disk_space propertylinux/amd64 build. Thanks to @christoffer-eidenever from choices of maintenance dowdevelopment flag to configure logger's behaviormake generate-user-configs)genericServiceHandler to generalize service managementKafkaACL deletionProjectVPCRef property to Kafka, OpenSearch, Clickhouse and Redis kinds to get ProjectVPC ID when resource is readyProjectVPC deletion, deletes by ID first if possible, then tries by nameclient.Object storage update data lossfeatures:
improvements:
fixes:
improvements:
fixes:
features:
Use the following checks to help you troubleshoot the Aiven Operator for Kubernetes.
"},{"location":"troubleshooting.html#check-the-pods","title":"Check the Pods","text":"Verify that all the operator Pods are READY, and the STATUS is Running.
kubectl get pod -n aiven-operator-system\nThe output is similar to the following:
NAME READY STATUS RESTARTS AGE\naiven-operator-controller-manager-576d944499-ggttj 1/1 Running 0 12m\nVerify that the cert-manager Pods are also running.
kubectl get pod -n cert-manager\nThe output has the status:
NAME READY STATUS RESTARTS AGE\ncert-manager-7dd5854bb4-85cpv 1/1 Running 0 76s\ncert-manager-cainjector-64c949654c-n2z8l 1/1 Running 0 77s\ncert-manager-webhook-6bdffc7c9d-47w6z 1/1 Running 0 76s\nUse the following command to visualize all the logs from the operator.
kubectl logs -n aiven-operator-system -l control-plane=controller-manager\nkubectl get pod -n aiven-operator-system -l control-plane=controller-manager -o jsonpath=\"{.items[0].spec.containers[0].image}\"\nWe're always working to resolve problems that pop up in Aiven products. If your problem is listed below, we know about it and are working to fix it. If your problem isn't listed below, report it as an issue.
"},{"location":"troubleshooting.html#cert-manager","title":"cert-manager","text":""},{"location":"troubleshooting.html#issue","title":"Issue","text":"The following event appears on the operator Pod:
MountVolume.SetUp failed for volume \"cert\" : secret \"webhook-server-cert\" not found\nYou cannot run the operator.
"},{"location":"troubleshooting.html#solution","title":"Solution","text":"Make sure that cert-manager is up and running.
kubectl get pod -n cert-manager\nThe output shows the status of each cert-manager:
NAME READY STATUS RESTARTS AGE\ncert-manager-7dd5854bb4-85cpv 1/1 Running 0 76s\ncert-manager-cainjector-64c949654c-n2z8l 1/1 Running 0 77s\ncert-manager-webhook-6bdffc7c9d-47w6z 1/1 Running 0 76s\nAutogenerated from CRD files.
"},{"location":"api-reference/cassandra.html","title":"Cassandra","text":""},{"location":"api-reference/cassandra.html#usage-example","title":"Usage example","text":"ExampleapiVersion: aiven.io/v1alpha1\nkind: Cassandra\nmetadata:\n name: my-cassandra\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n connInfoSecretTarget:\n name: cassandra-secret\n prefix: MY_SECRET_PREFIX_\n annotations:\n foo: bar\n labels:\n baz: egg\n\n project: aiven-project-name\n cloudName: google-europe-west1\n plan: startup-4\n\n maintenanceWindowDow: sunday\n maintenanceWindowTime: 11:00:00\n\n userConfig:\n migrate_sstableloader: true\n public_access:\n prometheus: true\n ip_filter:\n - network: 0.0.0.0\n description: whatever\n - network: 10.20.0.0/16\nInfo
To create this resource, a Secret containing Aiven token must be created first.
Apply the resource with:
kubectl apply -f example.yaml\nVerify the newly created Cassandra:
kubectl get cassandras my-cassandra\nThe output is similar to the following:
Name Project Region Plan State \nmy-cassandra aiven-project-name google-europe-west1 startup-4 RUNNING \nTo view the details of the Secret, use the following command:
kubectl describe secret cassandra-secret\nYou can use the jq to quickly decode the Secret:
kubectl get secret cassandra-secret -o json | jq '.data | map_values(@base64d)'\nThe output is similar to the following:
{\n \"CASSANDRA_HOST\": \"<secret>\",\n \"CASSANDRA_PORT\": \"<secret>\",\n \"CASSANDRA_USER\": \"<secret>\",\n \"CASSANDRA_PASSWORD\": \"<secret>\",\n \"CASSANDRA_URI\": \"<secret>\",\n \"CASSANDRA_HOSTS\": \"<secret>\",\n \"CASSANDRA_CA_CERT\": \"<secret>\",\n}\nCassandra is the Schema for the cassandras API.
Exposes secret keys
CASSANDRA_HOST, CASSANDRA_PORT, CASSANDRA_USER, CASSANDRA_PASSWORD, CASSANDRA_URI, CASSANDRA_HOSTS, CASSANDRA_CA_CERT.
Required
apiVersion (string). Value aiven.io/v1alpha1.kind (string). Value Cassandra.metadata (object). Data that identifies the object, including a name string and optional namespace.spec (object). CassandraSpec defines the desired state of Cassandra. See below for nested schema.Appears on Cassandra.
CassandraSpec defines the desired state of Cassandra.
Required
plan (string, MaxLength: 128). Subscription plan.project (string, Immutable, Pattern: ^[a-zA-Z0-9_-]+$, MaxLength: 63). Identifies the project this resource belongs to.Optional
authSecretRef (object). Authentication reference to Aiven token in a secret. See below for nested schema.cloudName (string, MaxLength: 256). Cloud the service runs in.connInfoSecretTarget (object). Secret configuration. See below for nested schema.connInfoSecretTargetDisabled (boolean, Immutable). When true, the secret containing connection information will not be created, defaults to false. This field cannot be changed after resource creation.disk_space (string, Pattern: (?i)^[1-9][0-9]*(GiB|G)?$). The disk space of the service, possible values depend on the service type, the cloud provider and the project. Reducing will result in the service re-balancing. The removal of this field does not change the value.maintenanceWindowDow (string, Enum: monday, tuesday, wednesday, thursday, friday, saturday, sunday). Day of week when maintenance operations should be performed. One monday, tuesday, wednesday, etc.maintenanceWindowTime (string, MaxLength: 8). Time of day when maintenance operations should be performed. UTC time in HH:mm:ss format.projectVPCRef (object). ProjectVPCRef reference to ProjectVPC resource to use its ID as ProjectVPCID automatically. See below for nested schema.projectVpcId (string, MaxLength: 36). Identifier of the VPC the service should be in, if any.serviceIntegrations (array of objects, Immutable, MaxItems: 1). Service integrations to specify when creating a service. Not applied after initial service creation. See below for nested schema.tags (object, AdditionalProperties: string). Tags are key-value pairs that allow you to categorize services.technicalEmails (array of objects, MaxItems: 10). Defines the email addresses that will receive alerts about upcoming maintenance updates or warnings about service instability. See below for nested schema.terminationProtection (boolean). Prevent service from being deleted. It is recommended to have this enabled for all services.userConfig (object). Cassandra specific user configuration options. See below for nested schema.Appears on spec.
Authentication reference to Aiven token in a secret.
Required
key (string, MinLength: 1).name (string, MinLength: 1).Appears on spec.
Secret configuration.
Required
name (string, Immutable). Name of the secret resource to be created. By default, it is equal to the resource name.Optional
annotations (object, AdditionalProperties: string). Annotations added to the secret.labels (object, AdditionalProperties: string). Labels added to the secret.prefix (string). Prefix for the secret's keys. Added \"as is\" without any transformations. By default, is equal to the kind name in uppercase + underscore, e.g. KAFKA_, REDIS_, etc.Appears on spec.
ProjectVPCRef reference to ProjectVPC resource to use its ID as ProjectVPCID automatically.
Required
name (string, MinLength: 1).Optional
namespace (string, MinLength: 1).Appears on spec.
Service integrations to specify when creating a service. Not applied after initial service creation.
Required
integrationType (string, Enum: read_replica).sourceServiceName (string, MinLength: 1, MaxLength: 64).Appears on spec.
Defines the email addresses that will receive alerts about upcoming maintenance updates or warnings about service instability.
Required
email (string). Email address.Appears on spec.
Cassandra specific user configuration options.
Optional
additional_backup_regions (array of strings, MaxItems: 1). Deprecated. Additional Cloud Regions for Backup Replication.backup_hour (integer, Minimum: 0, Maximum: 23). The hour of day (in UTC) when backup for the service is started. New backup is only started if previous backup has already completed.backup_minute (integer, Minimum: 0, Maximum: 59). The minute of an hour when backup for the service is started. New backup is only started if previous backup has already completed.cassandra (object). cassandra configuration values. See below for nested schema.cassandra_version (string, Enum: 4, 4.1, Pattern: ^[0-9]+(\\.[0-9]+)?$). Cassandra version.ip_filter (array of objects, MaxItems: 1024). Allow incoming connections from CIDR address block, e.g. 10.20.0.0/16. See below for nested schema.migrate_sstableloader (boolean). Sets the service into migration mode enabling the sstableloader utility to be used to upload Cassandra data files. Available only on service create.private_access (object). Allow access to selected service ports from private networks. See below for nested schema.project_to_fork_from (string, Immutable, Pattern: ^[a-z][-a-z0-9]{0,63}$|^$, MaxLength: 63). Name of another project to fork a service from. This has effect only when a new service is being created.public_access (object). Allow access to selected service ports from the public Internet. See below for nested schema.service_log (boolean). Store logs for the service so that they are available in the HTTP API and console.service_to_fork_from (string, Immutable, Pattern: ^[a-z][-a-z0-9]{0,63}$|^$, MaxLength: 64). Name of another service to fork from. This has effect only when a new service is being created.service_to_join_with (string, Pattern: ^[a-z][-a-z0-9]{0,63}$, MaxLength: 64). When bootstrapping, instead of creating a new Cassandra cluster try to join an existing one from another service. Can only be set on service creation.static_ips (boolean). Use static public IP addresses.Appears on spec.userConfig.
cassandra configuration values.
Optional
batch_size_fail_threshold_in_kb (integer, Minimum: 1, Maximum: 1000000). Fail any multiple-partition batch exceeding this value. 50kb (10x warn threshold) by default.batch_size_warn_threshold_in_kb (integer, Minimum: 1, Maximum: 1000000). Log a warning message on any multiple-partition batch size exceeding this value.5kb per batch by default.Caution should be taken on increasing the size of this thresholdas it can lead to node instability.datacenter (string, MaxLength: 128). Name of the datacenter to which nodes of this service belong. Can be set only when creating the service.read_request_timeout_in_ms (integer, Minimum: 1000, Maximum: 10000). How long the coordinator waits for read operations to complete before timing it out. 5 seconds by default.write_request_timeout_in_ms (integer, Minimum: 1000, Maximum: 10000). How long the coordinator waits for write requests to complete with at least one node in the local datacenter. 2 seconds by default.Appears on spec.userConfig.
CIDR address block, either as a string, or in a dict with an optional description field.
Required
network (string, MaxLength: 43). CIDR address block.Optional
description (string, MaxLength: 1024). Description for IP filter list entry.Appears on spec.userConfig.
Allow access to selected service ports from private networks.
Required
prometheus (boolean). Allow clients to connect to prometheus with a DNS name that always resolves to the service's private IP addresses. Only available in certain network locations.Appears on spec.userConfig.
Allow access to selected service ports from the public Internet.
Required
prometheus (boolean). Allow clients to connect to prometheus from the public internet for service nodes that are in a project VPC or another type of private network.apiVersion: aiven.io/v1alpha1\nkind: Clickhouse\nmetadata:\n name: my-clickhouse\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n connInfoSecretTarget:\n name: my-clickhouse\n annotations:\n foo: bar\n labels:\n baz: egg\n\n tags:\n env: test\n instance: foo\n\n userConfig:\n ip_filter:\n - network: 0.0.0.0/32\n description: bar\n - network: 10.20.0.0/16\n\n project: my-aiven-project\n cloudName: google-europe-west1\n plan: startup-16\n\n maintenanceWindowDow: friday\n maintenanceWindowTime: 23:00:00\nInfo
To create this resource, a Secret containing Aiven token must be created first.
Apply the resource with:
kubectl apply -f example.yaml\nVerify the newly created Clickhouse:
kubectl get clickhouses my-clickhouse\nThe output is similar to the following:
Name Project Region Plan State \nmy-clickhouse my-aiven-project google-europe-west1 startup-16 RUNNING \nTo view the details of the Secret, use the following command:
kubectl describe secret my-clickhouse\nYou can use the jq to quickly decode the Secret:
kubectl get secret my-clickhouse -o json | jq '.data | map_values(@base64d)'\nThe output is similar to the following:
{\n \"CLICKHOUSE_HOST\": \"<secret>\",\n \"CLICKHOUSE_PORT\": \"<secret>\",\n \"CLICKHOUSE_USER\": \"<secret>\",\n \"CLICKHOUSE_PASSWORD\": \"<secret>\",\n}\nClickhouse is the Schema for the clickhouses API.
Exposes secret keys
CLICKHOUSE_HOST, CLICKHOUSE_PORT, CLICKHOUSE_USER, CLICKHOUSE_PASSWORD.
Required
apiVersion (string). Value aiven.io/v1alpha1.kind (string). Value Clickhouse.metadata (object). Data that identifies the object, including a name string and optional namespace.spec (object). ClickhouseSpec defines the desired state of Clickhouse. See below for nested schema.Appears on Clickhouse.
ClickhouseSpec defines the desired state of Clickhouse.
Required
plan (string, MaxLength: 128). Subscription plan.project (string, Immutable, Pattern: ^[a-zA-Z0-9_-]+$, MaxLength: 63). Identifies the project this resource belongs to.Optional
authSecretRef (object). Authentication reference to Aiven token in a secret. See below for nested schema.cloudName (string, MaxLength: 256). Cloud the service runs in.connInfoSecretTarget (object). Secret configuration. See below for nested schema.connInfoSecretTargetDisabled (boolean, Immutable). When true, the secret containing connection information will not be created, defaults to false. This field cannot be changed after resource creation.disk_space (string, Pattern: (?i)^[1-9][0-9]*(GiB|G)?$). The disk space of the service, possible values depend on the service type, the cloud provider and the project. Reducing will result in the service re-balancing. The removal of this field does not change the value.maintenanceWindowDow (string, Enum: monday, tuesday, wednesday, thursday, friday, saturday, sunday). Day of week when maintenance operations should be performed. One monday, tuesday, wednesday, etc.maintenanceWindowTime (string, MaxLength: 8). Time of day when maintenance operations should be performed. UTC time in HH:mm:ss format.projectVPCRef (object). ProjectVPCRef reference to ProjectVPC resource to use its ID as ProjectVPCID automatically. See below for nested schema.projectVpcId (string, MaxLength: 36). Identifier of the VPC the service should be in, if any.serviceIntegrations (array of objects, Immutable, MaxItems: 1). Service integrations to specify when creating a service. Not applied after initial service creation. See below for nested schema.tags (object, AdditionalProperties: string). Tags are key-value pairs that allow you to categorize services.technicalEmails (array of objects, MaxItems: 10). Defines the email addresses that will receive alerts about upcoming maintenance updates or warnings about service instability. See below for nested schema.terminationProtection (boolean). Prevent service from being deleted. It is recommended to have this enabled for all services.userConfig (object). OpenSearch specific user configuration options. See below for nested schema.Appears on spec.
Authentication reference to Aiven token in a secret.
Required
key (string, MinLength: 1).name (string, MinLength: 1).Appears on spec.
Secret configuration.
Required
name (string, Immutable). Name of the secret resource to be created. By default, it is equal to the resource name.Optional
annotations (object, AdditionalProperties: string). Annotations added to the secret.labels (object, AdditionalProperties: string). Labels added to the secret.prefix (string). Prefix for the secret's keys. Added \"as is\" without any transformations. By default, is equal to the kind name in uppercase + underscore, e.g. KAFKA_, REDIS_, etc.Appears on spec.
ProjectVPCRef reference to ProjectVPC resource to use its ID as ProjectVPCID automatically.
Required
name (string, MinLength: 1).Optional
namespace (string, MinLength: 1).Appears on spec.
Service integrations to specify when creating a service. Not applied after initial service creation.
Required
integrationType (string, Enum: read_replica).sourceServiceName (string, MinLength: 1, MaxLength: 64).Appears on spec.
Defines the email addresses that will receive alerts about upcoming maintenance updates or warnings about service instability.
Required
email (string). Email address.Appears on spec.
OpenSearch specific user configuration options.
Optional
additional_backup_regions (array of strings, MaxItems: 1). Deprecated. Additional Cloud Regions for Backup Replication.ip_filter (array of objects, MaxItems: 1024). Allow incoming connections from CIDR address block, e.g. 10.20.0.0/16. See below for nested schema.private_access (object). Allow access to selected service ports from private networks. See below for nested schema.privatelink_access (object). Allow access to selected service components through Privatelink. See below for nested schema.project_to_fork_from (string, Immutable, Pattern: ^[a-z][-a-z0-9]{0,63}$|^$, MaxLength: 63). Name of another project to fork a service from. This has effect only when a new service is being created.public_access (object). Allow access to selected service ports from the public Internet. See below for nested schema.service_log (boolean). Store logs for the service so that they are available in the HTTP API and console.service_to_fork_from (string, Immutable, Pattern: ^[a-z][-a-z0-9]{0,63}$|^$, MaxLength: 64). Name of another service to fork from. This has effect only when a new service is being created.static_ips (boolean). Use static public IP addresses.Appears on spec.userConfig.
CIDR address block, either as a string, or in a dict with an optional description field.
Required
network (string, MaxLength: 43). CIDR address block.Optional
description (string, MaxLength: 1024). Description for IP filter list entry.Appears on spec.userConfig.
Allow access to selected service ports from private networks.
Optional
clickhouse (boolean). Allow clients to connect to clickhouse with a DNS name that always resolves to the service's private IP addresses. Only available in certain network locations.clickhouse_https (boolean). Allow clients to connect to clickhouse_https with a DNS name that always resolves to the service's private IP addresses. Only available in certain network locations.clickhouse_mysql (boolean). Allow clients to connect to clickhouse_mysql with a DNS name that always resolves to the service's private IP addresses. Only available in certain network locations.prometheus (boolean). Allow clients to connect to prometheus with a DNS name that always resolves to the service's private IP addresses. Only available in certain network locations.Appears on spec.userConfig.
Allow access to selected service components through Privatelink.
Optional
clickhouse (boolean). Enable clickhouse.clickhouse_https (boolean). Enable clickhouse_https.clickhouse_mysql (boolean). Enable clickhouse_mysql.prometheus (boolean). Enable prometheus.Appears on spec.userConfig.
Allow access to selected service ports from the public Internet.
Optional
clickhouse (boolean). Allow clients to connect to clickhouse from the public internet for service nodes that are in a project VPC or another type of private network.clickhouse_https (boolean). Allow clients to connect to clickhouse_https from the public internet for service nodes that are in a project VPC or another type of private network.clickhouse_mysql (boolean). Allow clients to connect to clickhouse_mysql from the public internet for service nodes that are in a project VPC or another type of private network.prometheus (boolean). Allow clients to connect to prometheus from the public internet for service nodes that are in a project VPC or another type of private network.apiVersion: aiven.io/v1alpha1\nkind: ClickhouseDatabase\nmetadata:\n name: my-db\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n project: my-aiven-project\n serviceName: my-clickhouse\n databaseName: example-db\nInfo
To create this resource, a Secret containing Aiven token must be created first.
Apply the resource with:
kubectl apply -f example.yaml\nVerify the newly created ClickhouseDatabase:
kubectl get clickhousedatabases my-db\nThe output is similar to the following:
Name Database name Service Name Project \nmy-db example-db my-clickhouse my-aiven-project \nClickhouseDatabase is the Schema for the databases API.
Required
apiVersion (string). Value aiven.io/v1alpha1.kind (string). Value ClickhouseDatabase.metadata (object). Data that identifies the object, including a name string and optional namespace.spec (object). ClickhouseDatabaseSpec defines the desired state of ClickhouseDatabase. See below for nested schema.Appears on ClickhouseDatabase.
ClickhouseDatabaseSpec defines the desired state of ClickhouseDatabase.
Required
project (string, Immutable, Pattern: ^[a-zA-Z0-9_-]+$, MaxLength: 63). Identifies the project this resource belongs to.serviceName (string, Immutable, Pattern: ^[a-z][-a-z0-9]+$, MaxLength: 63). Specifies the name of the service that this resource belongs to.Optional
authSecretRef (object). Authentication reference to Aiven token in a secret. See below for nested schema.databaseName (string, Immutable, MaxLength: 63). Specifies the Clickhouse database name. Defaults to metadata.name if omitted.Note
metadata.name is ASCII-only. For UTF-8 names, use spec.databaseName, but ASCII is advised for compatibility.
Appears on spec.
Authentication reference to Aiven token in a secret.
Required
key (string, MinLength: 1).name (string, MinLength: 1).apiVersion: aiven.io/v1alpha1\nkind: ClickhouseGrant\nmetadata:\n name: demo-ch-grant\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n project: my-aiven-project\n serviceName: my-clickhouse\n\n privilegeGrants:\n - grantees:\n - user: user1\n - user: my-clickhouse-user-\ud83e\udd84\n privileges:\n - SELECT\n - INSERT\n database: my-db\n # If table is omitted, the privileges are granted on all tables in the database\n # If columns is omitted, the privileges are granted on all columns in the table\n - grantees:\n - role: my-role\n privileges:\n - SELECT\n database: my-db\n table: my-table\n columns:\n - col1\n - col2\n\n roleGrants:\n - roles:\n - other-role\n grantees:\n - user: my-user\n - role: my-role\napiVersion: aiven.io/v1alpha1\nkind: ClickhouseGrant\nmetadata:\n name: my-clickhouse-grant\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n project: aiven-project-name\n serviceName: my-clickhouse-service\n\n privilegeGrants:\n - grantees:\n - role: my-clickhouse-role\n privileges:\n - INSERT\n - SELECT\n - CREATE TABLE\n - CREATE VIEW\n database: my-clickhouse-db\n roleGrants:\n - grantees:\n - user: my-clickhouse-user\n roles:\n - my-clickhouse-role\n\n---\n\napiVersion: aiven.io/v1alpha1\nkind: Clickhouse\nmetadata:\n name: my-clickhouse-service\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n project: aiven-project-name\n cloudName: google-europe-west1\n plan: startup-16\n\n---\n\napiVersion: aiven.io/v1alpha1\nkind: ClickhouseDatabase\nmetadata:\n name: my-clickhouse-db\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n project: aiven-project-name\n serviceName: my-clickhouse-service\n\n---\n\napiVersion: aiven.io/v1alpha1\nkind: ClickhouseUser\nmetadata:\n name: my-clickhouse-user\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n project: aiven-project-name\n serviceName: my-clickhouse-service\n\n---\n\napiVersion: aiven.io/v1alpha1\nkind: ClickhouseRole\nmetadata:\n name: my-clickhouse-role\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n project: aiven-project-name\n serviceName: my-clickhouse-service\n role: my-clickhouse-role\nInfo
To create this resource, a Secret containing Aiven token must be created first.
Apply the resource with:
kubectl apply -f example.yaml\nVerify the newly created ClickhouseGrant:
kubectl get clickhousegrants demo-ch-grant\nThe output is similar to the following:
Name Project Service Name \ndemo-ch-grant my-aiven-project my-clickhouse \nClickhouseGrant is the Schema for the ClickhouseGrants API
Warning
Due to the way ClickHouse operates, updating this resource first revokes the existing privileges.
Required
apiVersion (string). Value aiven.io/v1alpha1.kind (string). Value ClickhouseGrant.metadata (object). Data that identifies the object, including a name string and optional namespace.spec (object). ClickhouseGrantSpec defines the desired state of ClickhouseGrant. See below for nested schema.Appears on ClickhouseGrant.
ClickhouseGrantSpec defines the desired state of ClickhouseGrant.
Required
project (string, Immutable, Pattern: ^[a-zA-Z0-9_-]+$, MaxLength: 63). Identifies the project this resource belongs to.serviceName (string, Immutable, Pattern: ^[a-z][-a-z0-9]+$, MaxLength: 63). Specifies the name of the service that this resource belongs to.Optional
authSecretRef (object). Authentication reference to Aiven token in a secret. See below for nested schema.privilegeGrants (array of objects). Configuration to grant a privilege. Privileges not in the manifest are revoked. Existing privileges are retained; new ones are granted. See below for nested schema.roleGrants (array of objects). Configuration to grant a role. Role grants not in the manifest are revoked. Existing role grants are retained; new ones are granted. See below for nested schema.Appears on spec.
Authentication reference to Aiven token in a secret.
Required
key (string, MinLength: 1).name (string, MinLength: 1).Appears on spec.
PrivilegeGrant represents the privileges to be granted to users or roles. See.
Required
database (string). The database that the grant refers to.grantees (array of objects, MinItems: 1). List of grantees (users or roles) to grant the privilege to. See below for nested schema.privileges (array of strings). The privileges to grant, i.e. INSERT, SELECT. See.Optional
columns (array of strings). The column that the grant refers to.table (string). The tables that the grant refers to. To grant a privilege on all tables in a database, omit this field instead of writing table: \"*\".withGrantOption (boolean). If true, then the grantee (user or role) get the permission to execute the GRANT query. Users can grant privileges of the same scope they have and less. See.Appears on spec.privilegeGrants.
Grantee represents a user or a role to which privileges or roles are granted.
Ambiguity in the GRANT syntax
Due to an ambiguity in the GRANT syntax in ClickHouse, users and roles should not share the same name. It is unclear whether a grant applies to the user or the role.
Optional
role (string).user (string).Appears on spec.
RoleGrant represents the roles to be assigned to users or roles. See.
Required
grantees (array of objects, MinItems: 1). List of grantees (users or roles) to grant the privilege to. See below for nested schema.roles (array of strings, MinItems: 1). List of roles to grant to the grantees.Optional
withAdminOption (boolean). If true, the grant is executed with ADMIN OPTION privilege. See.Appears on spec.roleGrants.
Grantee represents a user or a role to which privileges or roles are granted.
Ambiguity in the GRANT syntax
Due to an ambiguity in the GRANT syntax in ClickHouse, users and roles should not share the same name. It is unclear whether a grant applies to the user or the role.
Optional
role (string).user (string).apiVersion: aiven.io/v1alpha1\nkind: ClickhouseRole\nmetadata:\n name: my-role\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n project: my-aiven-project\n serviceName: my-clickhouse\n role: my-role\nInfo
To create this resource, a Secret containing Aiven token must be created first.
Apply the resource with:
kubectl apply -f example.yaml\nVerify the newly created ClickhouseRole:
kubectl get clickhouseroles my-role\nThe output is similar to the following:
Name Project Service Name Role \nmy-role my-aiven-project my-clickhouse my-role \nClickhouseRole is the Schema for the clickhouseroles API.
Required
apiVersion (string). Value aiven.io/v1alpha1.kind (string). Value ClickhouseRole.metadata (object). Data that identifies the object, including a name string and optional namespace.spec (object). ClickhouseRoleSpec defines the desired state of ClickhouseRole. See below for nested schema.Appears on ClickhouseRole.
ClickhouseRoleSpec defines the desired state of ClickhouseRole.
Required
project (string, Immutable, Pattern: ^[a-zA-Z0-9_-]+$, MaxLength: 63). Identifies the project this resource belongs to.role (string, Immutable, MaxLength: 255). The role that is to be created.serviceName (string, Immutable, Pattern: ^[a-z][-a-z0-9]+$, MaxLength: 63). Specifies the name of the service that this resource belongs to.Optional
authSecretRef (object). Authentication reference to Aiven token in a secret. See below for nested schema.Appears on spec.
Authentication reference to Aiven token in a secret.
Required
key (string, MinLength: 1).name (string, MinLength: 1).apiVersion: aiven.io/v1alpha1\nkind: ClickhouseUser\nmetadata:\n name: my-clickhouse-user\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n connInfoSecretTarget:\n name: clickhouse-user-secret\n annotations:\n foo: bar\n labels:\n baz: egg\n\n project: my-aiven-project\n serviceName: my-clickhouse\n username: example-username\n\n---\n\napiVersion: aiven.io/v1alpha1\nkind: Clickhouse\nmetadata:\n name: my-clickhouse\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n project: my-aiven-project\n cloudName: google-europe-west1\n plan: startup-16\nInfo
To create this resource, a Secret containing Aiven token must be created first.
Apply the resource with:
kubectl apply -f example.yaml\nVerify the newly created ClickhouseUser:
kubectl get clickhouseusers my-clickhouse-user\nThe output is similar to the following:
Name Username Service Name Project \nmy-clickhouse-user example-username my-clickhouse my-aiven-project \nTo view the details of the Secret, use the following command:
kubectl describe secret clickhouse-user-secret\nYou can use the jq to quickly decode the Secret:
kubectl get secret clickhouse-user-secret -o json | jq '.data | map_values(@base64d)'\nThe output is similar to the following:
{\n \"CLICKHOUSEUSER_HOST\": \"<secret>\",\n \"CLICKHOUSEUSER_PORT\": \"<secret>\",\n \"CLICKHOUSEUSER_USER\": \"<secret>\",\n \"CLICKHOUSEUSER_PASSWORD\": \"<secret>\",\n}\nClickhouseUser is the Schema for the clickhouseusers API.
Exposes secret keys
CLICKHOUSEUSER_HOST, CLICKHOUSEUSER_PORT, CLICKHOUSEUSER_USER, CLICKHOUSEUSER_PASSWORD.
Required
apiVersion (string). Value aiven.io/v1alpha1.kind (string). Value ClickhouseUser.metadata (object). Data that identifies the object, including a name string and optional namespace.spec (object). ClickhouseUserSpec defines the desired state of ClickhouseUser. See below for nested schema.Appears on ClickhouseUser.
ClickhouseUserSpec defines the desired state of ClickhouseUser.
Required
project (string, Immutable, Pattern: ^[a-zA-Z0-9_-]+$, MaxLength: 63). Identifies the project this resource belongs to.serviceName (string, Immutable, Pattern: ^[a-z][-a-z0-9]+$, MaxLength: 63). Specifies the name of the service that this resource belongs to.Optional
authSecretRef (object). Authentication reference to Aiven token in a secret. See below for nested schema.connInfoSecretTarget (object). Secret configuration. See below for nested schema.connInfoSecretTargetDisabled (boolean, Immutable). When true, the secret containing connection information will not be created, defaults to false. This field cannot be changed after resource creation.username (string, Immutable, MaxLength: 63). Name of the Clickhouse user. Defaults to metadata.name if omitted.Note
metadata.name is ASCII-only. For UTF-8 names, use spec.username, but ASCII is advised for compatibility.
Appears on spec.
Authentication reference to Aiven token in a secret.
Required
key (string, MinLength: 1).name (string, MinLength: 1).Appears on spec.
Secret configuration.
Required
name (string, Immutable). Name of the secret resource to be created. By default, it is equal to the resource name.Optional
annotations (object, AdditionalProperties: string). Annotations added to the secret.labels (object, AdditionalProperties: string). Labels added to the secret.prefix (string). Prefix for the secret's keys. Added \"as is\" without any transformations. By default, is equal to the kind name in uppercase + underscore, e.g. KAFKA_, REDIS_, etc.apiVersion: aiven.io/v1alpha1\nkind: ConnectionPool\nmetadata:\n name: my-connection-pool\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n project: aiven-project-name\n serviceName: my-pg\n databaseName: my-database\n username: my-service-user\n poolMode: transaction\n poolSize: 25\n\n---\n\napiVersion: aiven.io/v1alpha1\nkind: PostgreSQL\nmetadata:\n name: my-pg\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n project: aiven-project-name\n cloudName: google-europe-west1\n plan: startup-4\n\n---\n\napiVersion: aiven.io/v1alpha1\nkind: Database\nmetadata:\n name: my-database\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n project: aiven-project-name\n serviceName: my-pg\n\n---\n\napiVersion: aiven.io/v1alpha1\nkind: ServiceUser\nmetadata:\n name: my-service-user\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n project: aiven-project-name\n serviceName: my-pg\nInfo
To create this resource, a Secret containing Aiven token must be created first.
Apply the resource with:
kubectl apply -f example.yaml\nVerify the newly created ConnectionPool:
kubectl get connectionpools my-connection-pool\nThe output is similar to the following:
Name Service Name Project Database Username Pool Size Pool Mode \nmy-connection-pool my-pg aiven-project-name my-database my-service-user 25 transaction \nConnectionPool is the Schema for the connectionpools API.
Exposes secret keys
CONNECTIONPOOL_NAME, CONNECTIONPOOL_HOST, CONNECTIONPOOL_PORT, CONNECTIONPOOL_DATABASE, CONNECTIONPOOL_USER, CONNECTIONPOOL_PASSWORD, CONNECTIONPOOL_SSLMODE, CONNECTIONPOOL_DATABASE_URI, CONNECTIONPOOL_CA_CERT.
Required
apiVersion (string). Value aiven.io/v1alpha1.kind (string). Value ConnectionPool.metadata (object). Data that identifies the object, including a name string and optional namespace.spec (object). ConnectionPoolSpec defines the desired state of ConnectionPool. See below for nested schema.Appears on ConnectionPool.
ConnectionPoolSpec defines the desired state of ConnectionPool.
Required
databaseName (string, MaxLength: 40). Name of the database the pool connects to.project (string, Immutable, Pattern: ^[a-zA-Z0-9_-]+$, MaxLength: 63). Identifies the project this resource belongs to.serviceName (string, Immutable, Pattern: ^[a-z][-a-z0-9]+$, MaxLength: 63). Specifies the name of the service that this resource belongs to.username (string, MaxLength: 64). Name of the service user used to connect to the database.Optional
authSecretRef (object). Authentication reference to Aiven token in a secret. See below for nested schema.connInfoSecretTarget (object). Secret configuration. See below for nested schema.connInfoSecretTargetDisabled (boolean, Immutable). When true, the secret containing connection information will not be created, defaults to false. This field cannot be changed after resource creation.poolMode (string, Enum: session, transaction, statement). Mode the pool operates in (session, transaction, statement).poolSize (integer). Number of connections the pool may create towards the backend server.Appears on spec.
Authentication reference to Aiven token in a secret.
Required
key (string, MinLength: 1).name (string, MinLength: 1).Appears on spec.
Secret configuration.
Required
name (string, Immutable). Name of the secret resource to be created. By default, it is equal to the resource name.Optional
annotations (object, AdditionalProperties: string). Annotations added to the secret.labels (object, AdditionalProperties: string). Labels added to the secret.prefix (string). Prefix for the secret's keys. Added \"as is\" without any transformations. By default, is equal to the kind name in uppercase + underscore, e.g. KAFKA_, REDIS_, etc.apiVersion: aiven.io/v1alpha1\nkind: Database\nmetadata:\n name: my-db\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n project: aiven-project-name\n serviceName: my-service\n\n lcCtype: en_US.UTF-8\n lcCollate: en_US.UTF-8\nInfo
To create this resource, a Secret containing Aiven token must be created first.
Apply the resource with:
kubectl apply -f example.yaml\nVerify the newly created Database:
kubectl get databases my-db\nThe output is similar to the following:
Name Project Service Name \nmy-db aiven-project-name my-service \nDatabase is the Schema for the databases API.
Required
apiVersion (string). Value aiven.io/v1alpha1.kind (string). Value Database.metadata (object). Data that identifies the object, including a name string and optional namespace.spec (object). DatabaseSpec defines the desired state of Database. See below for nested schema.Appears on Database.
DatabaseSpec defines the desired state of Database.
Required
project (string, Immutable, Pattern: ^[a-zA-Z0-9_-]+$, MaxLength: 63). Identifies the project this resource belongs to.serviceName (string, Immutable, Pattern: ^[a-z][-a-z0-9]+$, MaxLength: 63). Specifies the name of the service that this resource belongs to.Optional
authSecretRef (object). Authentication reference to Aiven token in a secret. See below for nested schema.lcCollate (string, Immutable, MaxLength: 128). Default string sort order (LC_COLLATE) of the database. Default value: en_US.UTF-8.lcCtype (string, Immutable, MaxLength: 128). Default character classification (LC_CTYPE) of the database. Default value: en_US.UTF-8.terminationProtection (boolean). It is a Kubernetes side deletion protections, which prevents the database from being deleted by Kubernetes. It is recommended to enable this for any production databases containing critical data.Appears on spec.
Authentication reference to Aiven token in a secret.
Required
key (string, MinLength: 1).name (string, MinLength: 1).apiVersion: aiven.io/v1alpha1\nkind: Grafana\nmetadata:\n name: my-grafana\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n connInfoSecretTarget:\n name: grafana-secret\n prefix: MY_SECRET_PREFIX_\n annotations:\n foo: bar\n labels:\n baz: egg\n\n project: my-aiven-project\n cloudName: google-europe-west1\n plan: startup-1\n\n maintenanceWindowDow: sunday\n maintenanceWindowTime: 11:00:00\n\n userConfig:\n public_access:\n grafana: true\n ip_filter:\n - network: 0.0.0.0\n description: whatever\n - network: 10.20.0.0/16\nInfo
To create this resource, a Secret containing Aiven token must be created first.
Apply the resource with:
kubectl apply -f example.yaml\nVerify the newly created Grafana:
kubectl get grafanas my-grafana\nThe output is similar to the following:
Name Project Region Plan State \nmy-grafana my-aiven-project google-europe-west1 startup-1 RUNNING \nTo view the details of the Secret, use the following command:
kubectl describe secret grafana-secret\nYou can use the jq to quickly decode the Secret:
kubectl get secret grafana-secret -o json | jq '.data | map_values(@base64d)'\nThe output is similar to the following:
{\n \"GRAFANA_HOST\": \"<secret>\",\n \"GRAFANA_PORT\": \"<secret>\",\n \"GRAFANA_USER\": \"<secret>\",\n \"GRAFANA_PASSWORD\": \"<secret>\",\n \"GRAFANA_URI\": \"<secret>\",\n \"GRAFANA_HOSTS\": \"<secret>\",\n}\nGrafana is the Schema for the grafanas API.
Exposes secret keys
GRAFANA_HOST, GRAFANA_PORT, GRAFANA_USER, GRAFANA_PASSWORD, GRAFANA_URI, GRAFANA_HOSTS.
Required
apiVersion (string). Value aiven.io/v1alpha1.kind (string). Value Grafana.metadata (object). Data that identifies the object, including a name string and optional namespace.spec (object). GrafanaSpec defines the desired state of Grafana. See below for nested schema.Appears on Grafana.
GrafanaSpec defines the desired state of Grafana.
Required
plan (string, MaxLength: 128). Subscription plan.project (string, Immutable, Pattern: ^[a-zA-Z0-9_-]+$, MaxLength: 63). Identifies the project this resource belongs to.Optional
authSecretRef (object). Authentication reference to Aiven token in a secret. See below for nested schema.cloudName (string, MaxLength: 256). Cloud the service runs in.connInfoSecretTarget (object). Secret configuration. See below for nested schema.connInfoSecretTargetDisabled (boolean, Immutable). When true, the secret containing connection information will not be created, defaults to false. This field cannot be changed after resource creation.disk_space (string, Pattern: (?i)^[1-9][0-9]*(GiB|G)?$). The disk space of the service, possible values depend on the service type, the cloud provider and the project. Reducing will result in the service re-balancing. The removal of this field does not change the value.maintenanceWindowDow (string, Enum: monday, tuesday, wednesday, thursday, friday, saturday, sunday). Day of week when maintenance operations should be performed. One monday, tuesday, wednesday, etc.maintenanceWindowTime (string, MaxLength: 8). Time of day when maintenance operations should be performed. UTC time in HH:mm:ss format.projectVPCRef (object). ProjectVPCRef reference to ProjectVPC resource to use its ID as ProjectVPCID automatically. See below for nested schema.projectVpcId (string, MaxLength: 36). Identifier of the VPC the service should be in, if any.serviceIntegrations (array of objects, Immutable, MaxItems: 1). Service integrations to specify when creating a service. Not applied after initial service creation. See below for nested schema.tags (object, AdditionalProperties: string). Tags are key-value pairs that allow you to categorize services.technicalEmails (array of objects, MaxItems: 10). Defines the email addresses that will receive alerts about upcoming maintenance updates or warnings about service instability. See below for nested schema.terminationProtection (boolean). Prevent service from being deleted. It is recommended to have this enabled for all services.userConfig (object). Cassandra specific user configuration options. See below for nested schema.Appears on spec.
Authentication reference to Aiven token in a secret.
Required
key (string, MinLength: 1).name (string, MinLength: 1).Appears on spec.
Secret configuration.
Required
name (string, Immutable). Name of the secret resource to be created. By default, it is equal to the resource name.Optional
annotations (object, AdditionalProperties: string). Annotations added to the secret.labels (object, AdditionalProperties: string). Labels added to the secret.prefix (string). Prefix for the secret's keys. Added \"as is\" without any transformations. By default, is equal to the kind name in uppercase + underscore, e.g. KAFKA_, REDIS_, etc.Appears on spec.
ProjectVPCRef reference to ProjectVPC resource to use its ID as ProjectVPCID automatically.
Required
name (string, MinLength: 1).Optional
namespace (string, MinLength: 1).Appears on spec.
Service integrations to specify when creating a service. Not applied after initial service creation.
Required
integrationType (string, Enum: read_replica).sourceServiceName (string, MinLength: 1, MaxLength: 64).Appears on spec.
Defines the email addresses that will receive alerts about upcoming maintenance updates or warnings about service instability.
Required
email (string). Email address.Appears on spec.
Cassandra specific user configuration options.
Optional
additional_backup_regions (array of strings, MaxItems: 1). Additional Cloud Regions for Backup Replication.alerting_enabled (boolean). Enable or disable Grafana legacy alerting functionality. This should not be enabled with unified_alerting_enabled.alerting_error_or_timeout (string, Enum: alerting, keep_state). Default error or timeout setting for new alerting rules.alerting_max_annotations_to_keep (integer, Minimum: 0, Maximum: 1000000). Max number of alert annotations that Grafana stores. 0 (default) keeps all alert annotations.alerting_nodata_or_nullvalues (string, Enum: alerting, no_data, keep_state, ok). Default value for 'no data or null values' for new alerting rules.allow_embedding (boolean). Allow embedding Grafana dashboards with iframe/frame/object/embed tags. Disabled by default to limit impact of clickjacking.auth_azuread (object). Azure AD OAuth integration. See below for nested schema.auth_basic_enabled (boolean). Enable or disable basic authentication form, used by Grafana built-in login.auth_generic_oauth (object). Generic OAuth integration. See below for nested schema.auth_github (object). Github Auth integration. See below for nested schema.auth_gitlab (object). GitLab Auth integration. See below for nested schema.auth_google (object). Google Auth integration. See below for nested schema.cookie_samesite (string, Enum: lax, strict, none). Cookie SameSite attribute: strict prevents sending cookie for cross-site requests, effectively disabling direct linking from other sites to Grafana. lax is the default value.custom_domain (string, MaxLength: 255). Serve the web frontend using a custom CNAME pointing to the Aiven DNS name.dashboard_previews_enabled (boolean). This feature is new in Grafana 9 and is quite resource intensive. It may cause low-end plans to work more slowly while the dashboard previews are rendering.dashboards_min_refresh_interval (string, Pattern: ^[0-9]+(ms|s|m|h|d)$, MaxLength: 16). Signed sequence of decimal numbers, followed by a unit suffix (ms, s, m, h, d), e.g. 30s, 1h.dashboards_versions_to_keep (integer, Minimum: 1, Maximum: 100). Dashboard versions to keep per dashboard.dataproxy_send_user_header (boolean). Send X-Grafana-User header to data source.dataproxy_timeout (integer, Minimum: 15, Maximum: 90). Timeout for data proxy requests in seconds.date_formats (object). Grafana date format specifications. See below for nested schema.disable_gravatar (boolean). Set to true to disable gravatar. Defaults to false (gravatar is enabled).editors_can_admin (boolean). Editors can manage folders, teams and dashboards created by them.external_image_storage (object). External image store settings. See below for nested schema.google_analytics_ua_id (string, Pattern: ^(G|UA|YT|MO)-[a-zA-Z0-9-]+$, MaxLength: 64). Google Analytics ID.ip_filter (array of objects, MaxItems: 1024). Allow incoming connections from CIDR address block, e.g. 10.20.0.0/16. See below for nested schema.metrics_enabled (boolean). Enable Grafana /metrics endpoint.oauth_allow_insecure_email_lookup (boolean). Enforce user lookup based on email instead of the unique ID provided by the IdP.private_access (object). Allow access to selected service ports from private networks. See below for nested schema.privatelink_access (object). Allow access to selected service components through Privatelink. See below for nested schema.project_to_fork_from (string, Immutable, Pattern: ^[a-z][-a-z0-9]{0,63}$|^$, MaxLength: 63). Name of another project to fork a service from. This has effect only when a new service is being created.public_access (object). Allow access to selected service ports from the public Internet. See below for nested schema.recovery_basebackup_name (string, Pattern: ^[a-zA-Z0-9-_:.]+$, MaxLength: 128). Name of the basebackup to restore in forked service.service_log (boolean). Store logs for the service so that they are available in the HTTP API and console.service_to_fork_from (string, Immutable, Pattern: ^[a-z][-a-z0-9]{0,63}$|^$, MaxLength: 64). Name of another service to fork from. This has effect only when a new service is being created.smtp_server (object). SMTP server settings. See below for nested schema.static_ips (boolean). Use static public IP addresses.unified_alerting_enabled (boolean). Enable or disable Grafana unified alerting functionality. By default this is enabled and any legacy alerts will be migrated on upgrade to Grafana 9+. To stay on legacy alerting, set unified_alerting_enabled to false and alerting_enabled to true. See https://grafana.com/docs/grafana/latest/alerting/set-up/migrating-alerts/ for more details.user_auto_assign_org (boolean). Auto-assign new users on signup to main organization. Defaults to false.user_auto_assign_org_role (string, Enum: Viewer, Admin, Editor). Set role for new signups. Defaults to Viewer.viewers_can_edit (boolean). Users with view-only permission can edit but not save dashboards.Appears on spec.userConfig.
Azure AD OAuth integration.
Required
auth_url (string, MaxLength: 2048). Authorization URL.client_id (string, Pattern: ^[\\040-\\176]+$, MaxLength: 1024). Client ID from provider.client_secret (string, Pattern: ^[\\040-\\176]+$, MaxLength: 1024). Client secret from provider.token_url (string, MaxLength: 2048). Token URL.Optional
allow_sign_up (boolean). Automatically sign-up users on successful sign-in.allowed_domains (array of strings, MaxItems: 50). Allowed domains.allowed_groups (array of strings, MaxItems: 50). Require users to belong to one of given groups.Appears on spec.userConfig.
Generic OAuth integration.
Required
api_url (string, MaxLength: 2048). API URL.auth_url (string, MaxLength: 2048). Authorization URL.client_id (string, Pattern: ^[\\040-\\176]+$, MaxLength: 1024). Client ID from provider.client_secret (string, Pattern: ^[\\040-\\176]+$, MaxLength: 1024). Client secret from provider.token_url (string, MaxLength: 2048). Token URL.Optional
allow_sign_up (boolean). Automatically sign-up users on successful sign-in.allowed_domains (array of strings, MaxItems: 50). Allowed domains.allowed_organizations (array of strings, MaxItems: 50). Require user to be member of one of the listed organizations.auto_login (boolean). Allow users to bypass the login screen and automatically log in.name (string, Pattern: ^[a-zA-Z0-9_\\- ]+$, MaxLength: 128). Name of the OAuth integration.scopes (array of strings, MaxItems: 50). OAuth scopes.Appears on spec.userConfig.
Github Auth integration.
Required
client_id (string, Pattern: ^[\\040-\\176]+$, MaxLength: 1024). Client ID from provider.client_secret (string, Pattern: ^[\\040-\\176]+$, MaxLength: 1024). Client secret from provider.Optional
allow_sign_up (boolean). Automatically sign-up users on successful sign-in.allowed_organizations (array of strings, MaxItems: 50). Require users to belong to one of given organizations.auto_login (boolean). Allow users to bypass the login screen and automatically log in.skip_org_role_sync (boolean). Stop automatically syncing user roles.team_ids (array of integers, MaxItems: 50). Require users to belong to one of given team IDs.Appears on spec.userConfig.
GitLab Auth integration.
Required
allowed_groups (array of strings, MaxItems: 50). Require users to belong to one of given groups.client_id (string, Pattern: ^[\\040-\\176]+$, MaxLength: 1024). Client ID from provider.client_secret (string, Pattern: ^[\\040-\\176]+$, MaxLength: 1024). Client secret from provider.Optional
allow_sign_up (boolean). Automatically sign-up users on successful sign-in.api_url (string, MaxLength: 2048). API URL. This only needs to be set when using self hosted GitLab.auth_url (string, MaxLength: 2048). Authorization URL. This only needs to be set when using self hosted GitLab.token_url (string, MaxLength: 2048). Token URL. This only needs to be set when using self hosted GitLab.Appears on spec.userConfig.
Google Auth integration.
Required
allowed_domains (array of strings, MaxItems: 64). Domains allowed to sign-in to this Grafana.client_id (string, Pattern: ^[\\040-\\176]+$, MaxLength: 1024). Client ID from provider.client_secret (string, Pattern: ^[\\040-\\176]+$, MaxLength: 1024). Client secret from provider.Optional
allow_sign_up (boolean). Automatically sign-up users on successful sign-in.Appears on spec.userConfig.
Grafana date format specifications.
Optional
default_timezone (string, MaxLength: 64). Default time zone for user preferences. Value browser uses browser local time zone.full_date (string, MaxLength: 128). Moment.js style format string for cases where full date is shown.interval_day (string, MaxLength: 128). Moment.js style format string used when a time requiring day accuracy is shown.interval_hour (string, MaxLength: 128). Moment.js style format string used when a time requiring hour accuracy is shown.interval_minute (string, MaxLength: 128). Moment.js style format string used when a time requiring minute accuracy is shown.interval_month (string, MaxLength: 128). Moment.js style format string used when a time requiring month accuracy is shown.interval_second (string, MaxLength: 128). Moment.js style format string used when a time requiring second accuracy is shown.interval_year (string, MaxLength: 128). Moment.js style format string used when a time requiring year accuracy is shown.Appears on spec.userConfig.
External image store settings.
Required
access_key (string, Pattern: ^[A-Z0-9]+$, MaxLength: 4096). S3 access key. Requires permissions to the S3 bucket for the s3:PutObject and s3:PutObjectAcl actions.bucket_url (string, MaxLength: 2048). Bucket URL for S3.provider (string, Enum: s3). Provider type.secret_key (string, Pattern: ^[A-Za-z0-9/+=]+$, MaxLength: 4096). S3 secret key.Appears on spec.userConfig.
CIDR address block, either as a string, or in a dict with an optional description field.
Required
network (string, MaxLength: 43). CIDR address block.Optional
description (string, MaxLength: 1024). Description for IP filter list entry.Appears on spec.userConfig.
Allow access to selected service ports from private networks.
Required
grafana (boolean). Allow clients to connect to grafana with a DNS name that always resolves to the service's private IP addresses. Only available in certain network locations.Appears on spec.userConfig.
Allow access to selected service components through Privatelink.
Required
grafana (boolean). Enable grafana.Appears on spec.userConfig.
Allow access to selected service ports from the public Internet.
Required
grafana (boolean). Allow clients to connect to grafana from the public internet for service nodes that are in a project VPC or another type of private network.Appears on spec.userConfig.
SMTP server settings.
Required
from_address (string, MaxLength: 319). Address used for sending emails.host (string, MaxLength: 255). Server hostname or IP.port (integer, Minimum: 1, Maximum: 65535). SMTP server port.Optional
from_name (string, Pattern: ^[^\\x00-\\x1F]+$, MaxLength: 128). Name used in outgoing emails, defaults to Grafana.password (string, Pattern: ^[^\\x00-\\x1F]+$, MaxLength: 255). Password for SMTP authentication.skip_verify (boolean). Skip verifying server certificate. Defaults to false.starttls_policy (string, Enum: OpportunisticStartTLS, MandatoryStartTLS, NoStartTLS). Either OpportunisticStartTLS, MandatoryStartTLS or NoStartTLS. Default is OpportunisticStartTLS.username (string, Pattern: ^[^\\x00-\\x1F]+$, MaxLength: 255). Username for SMTP authentication.apiVersion: aiven.io/v1alpha1\nkind: Kafka\nmetadata:\n name: my-kafka\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n connInfoSecretTarget:\n name: kafka-secret\n prefix: MY_SECRET_PREFIX_\n annotations:\n foo: bar\n labels:\n baz: egg\n\n project: my-aiven-project\n cloudName: google-europe-west1\n plan: startup-2\n\n maintenanceWindowDow: friday\n maintenanceWindowTime: 23:00:00\nInfo
To create this resource, a Secret containing Aiven token must be created first.
Apply the resource with:
kubectl apply -f example.yaml\nVerify the newly created Kafka:
kubectl get kafkas my-kafka\nThe output is similar to the following:
Name Project Region Plan State \nmy-kafka my-aiven-project google-europe-west1 startup-2 RUNNING \nTo view the details of the Secret, use the following command:
kubectl describe secret kafka-secret\nYou can use the jq to quickly decode the Secret:
kubectl get secret kafka-secret -o json | jq '.data | map_values(@base64d)'\nThe output is similar to the following:
{\n \"KAFKA_HOST\": \"<secret>\",\n \"KAFKA_PORT\": \"<secret>\",\n \"KAFKA_USERNAME\": \"<secret>\",\n \"KAFKA_PASSWORD\": \"<secret>\",\n \"KAFKA_ACCESS_CERT\": \"<secret>\",\n \"KAFKA_ACCESS_KEY\": \"<secret>\",\n \"KAFKA_SASL_HOST\": \"<secret>\",\n \"KAFKA_SASL_PORT\": \"<secret>\",\n \"KAFKA_SCHEMA_REGISTRY_HOST\": \"<secret>\",\n \"KAFKA_SCHEMA_REGISTRY_PORT\": \"<secret>\",\n \"KAFKA_CONNECT_HOST\": \"<secret>\",\n \"KAFKA_CONNECT_PORT\": \"<secret>\",\n \"KAFKA_REST_HOST\": \"<secret>\",\n \"KAFKA_REST_PORT\": \"<secret>\",\n \"KAFKA_CA_CERT\": \"<secret>\",\n}\nKafka is the Schema for the kafkas API.
Exposes secret keys
KAFKA_HOST, KAFKA_PORT, KAFKA_USERNAME, KAFKA_PASSWORD, KAFKA_ACCESS_CERT, KAFKA_ACCESS_KEY, KAFKA_SASL_HOST, KAFKA_SASL_PORT, KAFKA_SCHEMA_REGISTRY_HOST, KAFKA_SCHEMA_REGISTRY_PORT, KAFKA_CONNECT_HOST, KAFKA_CONNECT_PORT, KAFKA_REST_HOST, KAFKA_REST_PORT, KAFKA_CA_CERT.
Required
apiVersion (string). Value aiven.io/v1alpha1.kind (string). Value Kafka.metadata (object). Data that identifies the object, including a name string and optional namespace.spec (object). KafkaSpec defines the desired state of Kafka. See below for nested schema.Appears on Kafka.
KafkaSpec defines the desired state of Kafka.
Required
plan (string, MaxLength: 128). Subscription plan.project (string, Immutable, Pattern: ^[a-zA-Z0-9_-]+$, MaxLength: 63). Identifies the project this resource belongs to.Optional
authSecretRef (object). Authentication reference to Aiven token in a secret. See below for nested schema.cloudName (string, MaxLength: 256). Cloud the service runs in.connInfoSecretTarget (object). Secret configuration. See below for nested schema.connInfoSecretTargetDisabled (boolean, Immutable). When true, the secret containing connection information will not be created, defaults to false. This field cannot be changed after resource creation.disk_space (string, Pattern: (?i)^[1-9][0-9]*(GiB|G)?$). The disk space of the service, possible values depend on the service type, the cloud provider and the project. Reducing will result in the service re-balancing. The removal of this field does not change the value.karapace (boolean). Switch the service to use Karapace for schema registry and REST proxy.maintenanceWindowDow (string, Enum: monday, tuesday, wednesday, thursday, friday, saturday, sunday). Day of week when maintenance operations should be performed. One monday, tuesday, wednesday, etc.maintenanceWindowTime (string, MaxLength: 8). Time of day when maintenance operations should be performed. UTC time in HH:mm:ss format.projectVPCRef (object). ProjectVPCRef reference to ProjectVPC resource to use its ID as ProjectVPCID automatically. See below for nested schema.projectVpcId (string, MaxLength: 36). Identifier of the VPC the service should be in, if any.serviceIntegrations (array of objects, Immutable, MaxItems: 1). Service integrations to specify when creating a service. Not applied after initial service creation. See below for nested schema.tags (object, AdditionalProperties: string). Tags are key-value pairs that allow you to categorize services.technicalEmails (array of objects, MaxItems: 10). Defines the email addresses that will receive alerts about upcoming maintenance updates or warnings about service instability. See below for nested schema.terminationProtection (boolean). Prevent service from being deleted. It is recommended to have this enabled for all services.userConfig (object). Kafka specific user configuration options. See below for nested schema.Appears on spec.
Authentication reference to Aiven token in a secret.
Required
key (string, MinLength: 1).name (string, MinLength: 1).Appears on spec.
Secret configuration.
Required
name (string, Immutable). Name of the secret resource to be created. By default, it is equal to the resource name.Optional
annotations (object, AdditionalProperties: string). Annotations added to the secret.labels (object, AdditionalProperties: string). Labels added to the secret.prefix (string). Prefix for the secret's keys. Added \"as is\" without any transformations. By default, is equal to the kind name in uppercase + underscore, e.g. KAFKA_, REDIS_, etc.Appears on spec.
ProjectVPCRef reference to ProjectVPC resource to use its ID as ProjectVPCID automatically.
Required
name (string, MinLength: 1).Optional
namespace (string, MinLength: 1).Appears on spec.
Service integrations to specify when creating a service. Not applied after initial service creation.
Required
integrationType (string, Enum: read_replica).sourceServiceName (string, MinLength: 1, MaxLength: 64).Appears on spec.
Defines the email addresses that will receive alerts about upcoming maintenance updates or warnings about service instability.
Required
email (string). Email address.Appears on spec.
Kafka specific user configuration options.
Optional
additional_backup_regions (array of strings, MaxItems: 1). Deprecated. Additional Cloud Regions for Backup Replication.aiven_kafka_topic_messages (boolean). Allow access to read Kafka topic messages in the Aiven Console and REST API.custom_domain (string, MaxLength: 255). Serve the web frontend using a custom CNAME pointing to the Aiven DNS name.ip_filter (array of objects, MaxItems: 1024). Allow incoming connections from CIDR address block, e.g. 10.20.0.0/16. See below for nested schema.kafka (object). Kafka broker configuration values. See below for nested schema.kafka_authentication_methods (object). Kafka authentication methods. See below for nested schema.kafka_connect (boolean). Enable Kafka Connect service.kafka_connect_config (object). Kafka Connect configuration values. See below for nested schema.kafka_connect_secret_providers. See below for } (array of objects). Configure external secret providers in order to reference external secrets in connector configuration. Currently Hashicorp Vault (provider: vault, auth_method: token) and AWS Secrets Manager (provider: aws, auth_method: credentials) are supported. Secrets can be referenced in connector config with ${::nested schema. kafka_rest (boolean). Enable Kafka-REST service.kafka_rest_authorization (boolean). Enable authorization in Kafka-REST service.kafka_rest_config (object). Kafka REST configuration. See below for nested schema.kafka_version (string, Enum: 3.4, 3.5, 3.6, 3.7). Kafka major version.letsencrypt_sasl_privatelink (boolean). Use Letsencrypt CA for Kafka SASL via Privatelink.private_access (object). Allow access to selected service ports from private networks. See below for nested schema.privatelink_access (object). Allow access to selected service components through Privatelink. See below for nested schema.public_access (object). Allow access to selected service ports from the public Internet. See below for nested schema.schema_registry (boolean). Enable Schema-Registry service.schema_registry_config (object). Schema Registry configuration. See below for nested schema.service_log (boolean). Store logs for the service so that they are available in the HTTP API and console.static_ips (boolean). Use static public IP addresses.tiered_storage (object). Tiered storage configuration. See below for nested schema.Appears on spec.userConfig.
CIDR address block, either as a string, or in a dict with an optional description field.
Required
network (string, MaxLength: 43). CIDR address block.Optional
description (string, MaxLength: 1024). Description for IP filter list entry.Appears on spec.userConfig.
Kafka broker configuration values.
Optional
auto_create_topics_enable (boolean). Enable auto creation of topics.compression_type (string, Enum: gzip, snappy, lz4, zstd, uncompressed, producer). Specify the final compression type for a given topic. This configuration accepts the standard compression codecs (gzip, snappy, lz4, zstd). It additionally accepts uncompressed which is equivalent to no compression; and producer which means retain the original compression codec set by the producer.connections_max_idle_ms (integer, Minimum: 1000, Maximum: 3600000). Idle connections timeout: the server socket processor threads close the connections that idle for longer than this.default_replication_factor (integer, Minimum: 1, Maximum: 10). Replication factor for autocreated topics.group_initial_rebalance_delay_ms (integer, Minimum: 0, Maximum: 300000). The amount of time, in milliseconds, the group coordinator will wait for more consumers to join a new group before performing the first rebalance. A longer delay means potentially fewer rebalances, but increases the time until processing begins. The default value for this is 3 seconds. During development and testing it might be desirable to set this to 0 in order to not delay test execution time.group_max_session_timeout_ms (integer, Minimum: 0, Maximum: 1800000). The maximum allowed session timeout for registered consumers. Longer timeouts give consumers more time to process messages in between heartbeats at the cost of a longer time to detect failures.group_min_session_timeout_ms (integer, Minimum: 0, Maximum: 60000). The minimum allowed session timeout for registered consumers. Longer timeouts give consumers more time to process messages in between heartbeats at the cost of a longer time to detect failures.log_cleaner_delete_retention_ms (integer, Minimum: 0, Maximum: 315569260000). How long are delete records retained?.log_cleaner_max_compaction_lag_ms (integer, Minimum: 30000). The maximum amount of time message will remain uncompacted. Only applicable for logs that are being compacted.log_cleaner_min_cleanable_ratio (number, Minimum: 0.2, Maximum: 0.9). Controls log compactor frequency. Larger value means more frequent compactions but also more space wasted for logs. Consider setting log.cleaner.max.compaction.lag.ms to enforce compactions sooner, instead of setting a very high value for this option.log_cleaner_min_compaction_lag_ms (integer, Minimum: 0). The minimum time a message will remain uncompacted in the log. Only applicable for logs that are being compacted.log_cleanup_policy (string, Enum: delete, compact, compact,delete). The default cleanup policy for segments beyond the retention window.log_flush_interval_messages (integer, Minimum: 1). The number of messages accumulated on a log partition before messages are flushed to disk.log_flush_interval_ms (integer, Minimum: 0). The maximum time in ms that a message in any topic is kept in memory before flushed to disk. If not set, the value in log.flush.scheduler.interval.ms is used.log_index_interval_bytes (integer, Minimum: 0, Maximum: 104857600). The interval with which Kafka adds an entry to the offset index.log_index_size_max_bytes (integer, Minimum: 1048576, Maximum: 104857600). The maximum size in bytes of the offset index.log_local_retention_bytes (integer, Minimum: -2). The maximum size of local log segments that can grow for a partition before it gets eligible for deletion. If set to -2, the value of log.retention.bytes is used. The effective value should always be less than or equal to log.retention.bytes value.log_local_retention_ms (integer, Minimum: -2). The number of milliseconds to keep the local log segments before it gets eligible for deletion. If set to -2, the value of log.retention.ms is used. The effective value should always be less than or equal to log.retention.ms value.log_message_downconversion_enable (boolean). This configuration controls whether down-conversion of message formats is enabled to satisfy consume requests.log_message_timestamp_difference_max_ms (integer, Minimum: 0). The maximum difference allowed between the timestamp when a broker receives a message and the timestamp specified in the message.log_message_timestamp_type (string, Enum: CreateTime, LogAppendTime). Define whether the timestamp in the message is message create time or log append time.log_preallocate (boolean). Should pre allocate file when create new segment?.log_retention_bytes (integer, Minimum: -1). The maximum size of the log before deleting messages.log_retention_hours (integer, Minimum: -1, Maximum: 2147483647). The number of hours to keep a log file before deleting it.log_retention_ms (integer, Minimum: -1). The number of milliseconds to keep a log file before deleting it (in milliseconds), If not set, the value in log.retention.minutes is used. If set to -1, no time limit is applied.log_roll_jitter_ms (integer, Minimum: 0). The maximum jitter to subtract from logRollTimeMillis (in milliseconds). If not set, the value in log.roll.jitter.hours is used.log_roll_ms (integer, Minimum: 1). The maximum time before a new log segment is rolled out (in milliseconds).log_segment_bytes (integer, Minimum: 10485760, Maximum: 1073741824). The maximum size of a single log file.log_segment_delete_delay_ms (integer, Minimum: 0, Maximum: 3600000). The amount of time to wait before deleting a file from the filesystem.max_connections_per_ip (integer, Minimum: 256, Maximum: 2147483647). The maximum number of connections allowed from each ip address (defaults to 2147483647).max_incremental_fetch_session_cache_slots (integer, Minimum: 1000, Maximum: 10000). The maximum number of incremental fetch sessions that the broker will maintain.message_max_bytes (integer, Minimum: 0, Maximum: 100001200). The maximum size of message that the server can receive.min_insync_replicas (integer, Minimum: 1, Maximum: 7). When a producer sets acks to all (or -1), min.insync.replicas specifies the minimum number of replicas that must acknowledge a write for the write to be considered successful.num_partitions (integer, Minimum: 1, Maximum: 1000). Number of partitions for autocreated topics.offsets_retention_minutes (integer, Minimum: 1, Maximum: 2147483647). Log retention window in minutes for offsets topic.producer_purgatory_purge_interval_requests (integer, Minimum: 10, Maximum: 10000). The purge interval (in number of requests) of the producer request purgatory(defaults to 1000).replica_fetch_max_bytes (integer, Minimum: 1048576, Maximum: 104857600). The number of bytes of messages to attempt to fetch for each partition (defaults to 1048576). This is not an absolute maximum, if the first record batch in the first non-empty partition of the fetch is larger than this value, the record batch will still be returned to ensure that progress can be made.replica_fetch_response_max_bytes (integer, Minimum: 10485760, Maximum: 1048576000). Maximum bytes expected for the entire fetch response (defaults to 10485760). Records are fetched in batches, and if the first record batch in the first non-empty partition of the fetch is larger than this value, the record batch will still be returned to ensure that progress can be made. As such, this is not an absolute maximum.sasl_oauthbearer_expected_audience (string, Pattern: ^[^\\r\\n]*$, MaxLength: 128). The (optional) comma-delimited setting for the broker to use to verify that the JWT was issued for one of the expected audiences.sasl_oauthbearer_expected_issuer (string, Pattern: ^[^\\r\\n]*$, MaxLength: 128). Optional setting for the broker to use to verify that the JWT was created by the expected issuer.sasl_oauthbearer_jwks_endpoint_url (string, MaxLength: 2048). OIDC JWKS endpoint URL. By setting this the SASL SSL OAuth2/OIDC authentication is enabled. See also other options for SASL OAuth2/OIDC.sasl_oauthbearer_sub_claim_name (string, Pattern: ^[^\\r\\n]*$, MaxLength: 128). Name of the scope from which to extract the subject claim from the JWT. Defaults to sub.socket_request_max_bytes (integer, Minimum: 10485760, Maximum: 209715200). The maximum number of bytes in a socket request (defaults to 104857600).transaction_partition_verification_enable (boolean). Enable verification that checks that the partition has been added to the transaction before writing transactional records to the partition.transaction_remove_expired_transaction_cleanup_interval_ms (integer, Minimum: 600000, Maximum: 3600000). The interval at which to remove transactions that have expired due to transactional.id.expiration.ms passing (defaults to 3600000 (1 hour)).transaction_state_log_segment_bytes (integer, Minimum: 1048576, Maximum: 2147483647). The transaction topic segment bytes should be kept relatively small in order to facilitate faster log compaction and cache loads (defaults to 104857600 (100 mebibytes)).Appears on spec.userConfig.
Kafka authentication methods.
Optional
certificate (boolean). Enable certificate/SSL authentication.sasl (boolean). Enable SASL authentication.Appears on spec.userConfig.
Kafka Connect configuration values.
Optional
connector_client_config_override_policy (string, Enum: None, All). Defines what client configurations can be overridden by the connector. Default is None.consumer_auto_offset_reset (string, Enum: earliest, latest). What to do when there is no initial offset in Kafka or if the current offset does not exist any more on the server. Default is earliest.consumer_fetch_max_bytes (integer, Minimum: 1048576, Maximum: 104857600). Records are fetched in batches by the consumer, and if the first record batch in the first non-empty partition of the fetch is larger than this value, the record batch will still be returned to ensure that the consumer can make progress. As such, this is not a absolute maximum.consumer_isolation_level (string, Enum: read_uncommitted, read_committed). Transaction read isolation level. read_uncommitted is the default, but read_committed can be used if consume-exactly-once behavior is desired.consumer_max_partition_fetch_bytes (integer, Minimum: 1048576, Maximum: 104857600). Records are fetched in batches by the consumer.If the first record batch in the first non-empty partition of the fetch is larger than this limit, the batch will still be returned to ensure that the consumer can make progress.consumer_max_poll_interval_ms (integer, Minimum: 1, Maximum: 2147483647). The maximum delay in milliseconds between invocations of poll() when using consumer group management (defaults to 300000).consumer_max_poll_records (integer, Minimum: 1, Maximum: 10000). The maximum number of records returned in a single call to poll() (defaults to 500).offset_flush_interval_ms (integer, Minimum: 1, Maximum: 100000000). The interval at which to try committing offsets for tasks (defaults to 60000).offset_flush_timeout_ms (integer, Minimum: 1, Maximum: 2147483647). Maximum number of milliseconds to wait for records to flush and partition offset data to be committed to offset storage before cancelling the process and restoring the offset data to be committed in a future attempt (defaults to 5000).producer_batch_size (integer, Minimum: 0, Maximum: 5242880). This setting gives the upper bound of the batch size to be sent. If there are fewer than this many bytes accumulated for this partition, the producer will linger for the linger.ms time waiting for more records to show up. A batch size of zero will disable batching entirely (defaults to 16384).producer_buffer_memory (integer, Minimum: 5242880, Maximum: 134217728). The total bytes of memory the producer can use to buffer records waiting to be sent to the broker (defaults to 33554432).producer_compression_type (string, Enum: gzip, snappy, lz4, zstd, none). Specify the default compression type for producers. This configuration accepts the standard compression codecs (gzip, snappy, lz4, zstd). It additionally accepts none which is the default and equivalent to no compression.producer_linger_ms (integer, Minimum: 0, Maximum: 5000). This setting gives the upper bound on the delay for batching: once there is batch.size worth of records for a partition it will be sent immediately regardless of this setting, however if there are fewer than this many bytes accumulated for this partition the producer will linger for the specified time waiting for more records to show up. Defaults to 0.producer_max_request_size (integer, Minimum: 131072, Maximum: 67108864). This setting will limit the number of record batches the producer will send in a single request to avoid sending huge requests.scheduled_rebalance_max_delay_ms (integer, Minimum: 0, Maximum: 600000). The maximum delay that is scheduled in order to wait for the return of one or more departed workers before rebalancing and reassigning their connectors and tasks to the group. During this period the connectors and tasks of the departed workers remain unassigned. Defaults to 5 minutes.session_timeout_ms (integer, Minimum: 1, Maximum: 2147483647). The timeout in milliseconds used to detect failures when using Kafka\u2019s group management facilities (defaults to 10000).Appears on spec.userConfig.
SecretProvider.
Required
name (string). Name of the secret provider. Used to reference secrets in connector config.Optional
aws (object). AWS config for Secret Provider. See below for nested schema.vault (object). Vault Config for Secret Provider. See below for nested schema.Appears on spec.userConfig.kafka_connect_secret_providers.
AWS config for Secret Provider.
Required
auth_method (string, Enum: credentials). Auth method of the vault secret provider.region (string, MaxLength: 64). Region used to lookup secrets with AWS SecretManager.Optional
access_key (string, MaxLength: 128). Access key used to authenticate with aws.secret_key (string, MaxLength: 128). Secret key used to authenticate with aws.Appears on spec.userConfig.kafka_connect_secret_providers.
Vault Config for Secret Provider.
Required
address (string, MinLength: 1, MaxLength: 65536). Address of the Vault server.auth_method (string, Enum: token). Auth method of the vault secret provider.Optional
engine_version (integer, Enum: 1, 2). KV Secrets Engine version of the Vault server instance.token (string, MaxLength: 256). Token used to authenticate with vault and auth method token.Appears on spec.userConfig.
Kafka REST configuration.
Optional
consumer_enable_auto_commit (boolean). If true the consumer's offset will be periodically committed to Kafka in the background.consumer_request_max_bytes (integer, Minimum: 0, Maximum: 671088640). Maximum number of bytes in unencoded message keys and values by a single request.consumer_request_timeout_ms (integer, Enum: 1000, 15000, 30000, Minimum: 1000, Maximum: 30000). The maximum total time to wait for messages for a request if the maximum number of messages has not yet been reached.name_strategy (string, Enum: topic_name, record_name, topic_record_name). Name strategy to use when selecting subject for storing schemas.name_strategy_validation (boolean). If true, validate that given schema is registered under expected subject name by the used name strategy when producing messages.producer_acks (string, Enum: all, -1, 0, 1). The number of acknowledgments the producer requires the leader to have received before considering a request complete. If set to all or -1, the leader will wait for the full set of in-sync replicas to acknowledge the record.producer_compression_type (string, Enum: gzip, snappy, lz4, zstd, none). Specify the default compression type for producers. This configuration accepts the standard compression codecs (gzip, snappy, lz4, zstd). It additionally accepts none which is the default and equivalent to no compression.producer_linger_ms (integer, Minimum: 0, Maximum: 5000). Wait for up to the given delay to allow batching records together.producer_max_request_size (integer, Minimum: 0, Maximum: 2147483647). The maximum size of a request in bytes. Note that Kafka broker can also cap the record batch size.simpleconsumer_pool_size_max (integer, Minimum: 10, Maximum: 250). Maximum number of SimpleConsumers that can be instantiated per broker.Appears on spec.userConfig.
Allow access to selected service ports from private networks.
Optional
kafka (boolean). Allow clients to connect to kafka with a DNS name that always resolves to the service's private IP addresses. Only available in certain network locations.kafka_connect (boolean). Allow clients to connect to kafka_connect with a DNS name that always resolves to the service's private IP addresses. Only available in certain network locations.kafka_rest (boolean). Allow clients to connect to kafka_rest with a DNS name that always resolves to the service's private IP addresses. Only available in certain network locations.prometheus (boolean). Allow clients to connect to prometheus with a DNS name that always resolves to the service's private IP addresses. Only available in certain network locations.schema_registry (boolean). Allow clients to connect to schema_registry with a DNS name that always resolves to the service's private IP addresses. Only available in certain network locations.Appears on spec.userConfig.
Allow access to selected service components through Privatelink.
Optional
jolokia (boolean). Enable jolokia.kafka (boolean). Enable kafka.kafka_connect (boolean). Enable kafka_connect.kafka_rest (boolean). Enable kafka_rest.prometheus (boolean). Enable prometheus.schema_registry (boolean). Enable schema_registry.Appears on spec.userConfig.
Allow access to selected service ports from the public Internet.
Optional
kafka (boolean). Allow clients to connect to kafka from the public internet for service nodes that are in a project VPC or another type of private network.kafka_connect (boolean). Allow clients to connect to kafka_connect from the public internet for service nodes that are in a project VPC or another type of private network.kafka_rest (boolean). Allow clients to connect to kafka_rest from the public internet for service nodes that are in a project VPC or another type of private network.prometheus (boolean). Allow clients to connect to prometheus from the public internet for service nodes that are in a project VPC or another type of private network.schema_registry (boolean). Allow clients to connect to schema_registry from the public internet for service nodes that are in a project VPC or another type of private network.Appears on spec.userConfig.
Schema Registry configuration.
Optional
leader_eligibility (boolean). If true, Karapace / Schema Registry on the service nodes can participate in leader election. It might be needed to disable this when the schemas topic is replicated to a secondary cluster and Karapace / Schema Registry there must not participate in leader election. Defaults to true.topic_name (string, MinLength: 1, MaxLength: 249). The durable single partition topic that acts as the durable log for the data. This topic must be compacted to avoid losing data due to retention policy. Please note that changing this configuration in an existing Schema Registry / Karapace setup leads to previous schemas being inaccessible, data encoded with them potentially unreadable and schema ID sequence put out of order. It's only possible to do the switch while Schema Registry / Karapace is disabled. Defaults to _schemas.Appears on spec.userConfig.
Tiered storage configuration.
Optional
enabled (boolean). Whether to enable the tiered storage functionality.local_cache (object). Deprecated. Local cache configuration. See below for nested schema.Appears on spec.userConfig.tiered_storage.
Deprecated. Local cache configuration.
Required
size (integer, Minimum: 1, Maximum: 107374182400). Deprecated. Local cache size in bytes.apiVersion: aiven.io/v1alpha1\nkind: KafkaACL\nmetadata:\n name: my-kafka-acl\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n project: my-aiven-project\n serviceName: my-kafka\n topic: my-topic\n username: my-user\n permission: admin\nInfo
To create this resource, a Secret containing Aiven token must be created first.
Apply the resource with:
kubectl apply -f example.yaml\nVerify the newly created KafkaACL:
kubectl get kafkaacls my-kafka-acl\nThe output is similar to the following:
Name Service Name Project Username Permission Topic \nmy-kafka-acl my-kafka my-aiven-project my-user admin my-topic \nKafkaACL is the Schema for the kafkaacls API.
Required
apiVersion (string). Value aiven.io/v1alpha1.kind (string). Value KafkaACL.metadata (object). Data that identifies the object, including a name string and optional namespace.spec (object). KafkaACLSpec defines the desired state of KafkaACL. See below for nested schema.Appears on KafkaACL.
KafkaACLSpec defines the desired state of KafkaACL.
Required
permission (string, Enum: admin, read, readwrite, write). Kafka permission to grant (admin, read, readwrite, write).project (string, Immutable, Pattern: ^[a-zA-Z0-9_-]+$, MaxLength: 63). Identifies the project this resource belongs to.serviceName (string, Immutable, Pattern: ^[a-z][-a-z0-9]+$, MaxLength: 63). Specifies the name of the service that this resource belongs to.topic (string). Topic name pattern for the ACL entry.username (string). Username pattern for the ACL entry.Optional
authSecretRef (object). Authentication reference to Aiven token in a secret. See below for nested schema.Appears on spec.
Authentication reference to Aiven token in a secret.
Required
key (string, MinLength: 1).name (string, MinLength: 1).apiVersion: aiven.io/v1alpha1\nkind: KafkaConnect\nmetadata:\n name: my-kafka-connect\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n project: my-aiven-project\n cloudName: google-europe-west1\n plan: business-4\n\n userConfig:\n kafka_connect:\n consumer_isolation_level: read_committed\n public_access:\n kafka_connect: true\nInfo
To create this resource, a Secret containing Aiven token must be created first.
Apply the resource with:
kubectl apply -f example.yaml\nVerify the newly created KafkaConnect:
kubectl get kafkaconnects my-kafka-connect\nThe output is similar to the following:
Name Project Region Plan State \nmy-kafka-connect my-aiven-project google-europe-west1 business-4 RUNNING \nKafkaConnect is the Schema for the kafkaconnects API.
Required
apiVersion (string). Value aiven.io/v1alpha1.kind (string). Value KafkaConnect.metadata (object). Data that identifies the object, including a name string and optional namespace.spec (object). KafkaConnectSpec defines the desired state of KafkaConnect. See below for nested schema.Appears on KafkaConnect.
KafkaConnectSpec defines the desired state of KafkaConnect.
Required
plan (string, MaxLength: 128). Subscription plan.project (string, Immutable, Pattern: ^[a-zA-Z0-9_-]+$, MaxLength: 63). Identifies the project this resource belongs to.Optional
authSecretRef (object). Authentication reference to Aiven token in a secret. See below for nested schema.cloudName (string, MaxLength: 256). Cloud the service runs in.maintenanceWindowDow (string, Enum: monday, tuesday, wednesday, thursday, friday, saturday, sunday). Day of week when maintenance operations should be performed. One monday, tuesday, wednesday, etc.maintenanceWindowTime (string, MaxLength: 8). Time of day when maintenance operations should be performed. UTC time in HH:mm:ss format.projectVPCRef (object). ProjectVPCRef reference to ProjectVPC resource to use its ID as ProjectVPCID automatically. See below for nested schema.projectVpcId (string, MaxLength: 36). Identifier of the VPC the service should be in, if any.serviceIntegrations (array of objects, Immutable, MaxItems: 1). Service integrations to specify when creating a service. Not applied after initial service creation. See below for nested schema.tags (object, AdditionalProperties: string). Tags are key-value pairs that allow you to categorize services.technicalEmails (array of objects, MaxItems: 10). Defines the email addresses that will receive alerts about upcoming maintenance updates or warnings about service instability. See below for nested schema.terminationProtection (boolean). Prevent service from being deleted. It is recommended to have this enabled for all services.userConfig (object). KafkaConnect specific user configuration options. See below for nested schema.Appears on spec.
Authentication reference to Aiven token in a secret.
Required
key (string, MinLength: 1).name (string, MinLength: 1).Appears on spec.
ProjectVPCRef reference to ProjectVPC resource to use its ID as ProjectVPCID automatically.
Required
name (string, MinLength: 1).Optional
namespace (string, MinLength: 1).Appears on spec.
Service integrations to specify when creating a service. Not applied after initial service creation.
Required
integrationType (string, Enum: read_replica).sourceServiceName (string, MinLength: 1, MaxLength: 64).Appears on spec.
Defines the email addresses that will receive alerts about upcoming maintenance updates or warnings about service instability.
Required
email (string). Email address.Appears on spec.
KafkaConnect specific user configuration options.
Optional
additional_backup_regions (array of strings, MaxItems: 1). Deprecated. Additional Cloud Regions for Backup Replication.ip_filter (array of objects, MaxItems: 1024). Allow incoming connections from CIDR address block, e.g. 10.20.0.0/16. See below for nested schema.kafka_connect (object). Kafka Connect configuration values. See below for nested schema.private_access (object). Allow access to selected service ports from private networks. See below for nested schema.privatelink_access (object). Allow access to selected service components through Privatelink. See below for nested schema.public_access (object). Allow access to selected service ports from the public Internet. See below for nested schema.secret_providers. See below for } (array of objects). Configure external secret providers in order to reference external secrets in connector configuration. Currently Hashicorp Vault (provider: vault, auth_method: token) and AWS Secrets Manager (provider: aws, auth_method: credentials) are supported. Secrets can be referenced in connector config with ${::nested schema. service_log (boolean). Store logs for the service so that they are available in the HTTP API and console.static_ips (boolean). Use static public IP addresses.Appears on spec.userConfig.
CIDR address block, either as a string, or in a dict with an optional description field.
Required
network (string, MaxLength: 43). CIDR address block.Optional
description (string, MaxLength: 1024). Description for IP filter list entry.Appears on spec.userConfig.
Kafka Connect configuration values.
Optional
connector_client_config_override_policy (string, Enum: None, All). Defines what client configurations can be overridden by the connector. Default is None.consumer_auto_offset_reset (string, Enum: earliest, latest). What to do when there is no initial offset in Kafka or if the current offset does not exist any more on the server. Default is earliest.consumer_fetch_max_bytes (integer, Minimum: 1048576, Maximum: 104857600). Records are fetched in batches by the consumer, and if the first record batch in the first non-empty partition of the fetch is larger than this value, the record batch will still be returned to ensure that the consumer can make progress. As such, this is not a absolute maximum.consumer_isolation_level (string, Enum: read_uncommitted, read_committed). Transaction read isolation level. read_uncommitted is the default, but read_committed can be used if consume-exactly-once behavior is desired.consumer_max_partition_fetch_bytes (integer, Minimum: 1048576, Maximum: 104857600). Records are fetched in batches by the consumer.If the first record batch in the first non-empty partition of the fetch is larger than this limit, the batch will still be returned to ensure that the consumer can make progress.consumer_max_poll_interval_ms (integer, Minimum: 1, Maximum: 2147483647). The maximum delay in milliseconds between invocations of poll() when using consumer group management (defaults to 300000).consumer_max_poll_records (integer, Minimum: 1, Maximum: 10000). The maximum number of records returned in a single call to poll() (defaults to 500).offset_flush_interval_ms (integer, Minimum: 1, Maximum: 100000000). The interval at which to try committing offsets for tasks (defaults to 60000).offset_flush_timeout_ms (integer, Minimum: 1, Maximum: 2147483647). Maximum number of milliseconds to wait for records to flush and partition offset data to be committed to offset storage before cancelling the process and restoring the offset data to be committed in a future attempt (defaults to 5000).producer_batch_size (integer, Minimum: 0, Maximum: 5242880). This setting gives the upper bound of the batch size to be sent. If there are fewer than this many bytes accumulated for this partition, the producer will linger for the linger.ms time waiting for more records to show up. A batch size of zero will disable batching entirely (defaults to 16384).producer_buffer_memory (integer, Minimum: 5242880, Maximum: 134217728). The total bytes of memory the producer can use to buffer records waiting to be sent to the broker (defaults to 33554432).producer_compression_type (string, Enum: gzip, snappy, lz4, zstd, none). Specify the default compression type for producers. This configuration accepts the standard compression codecs (gzip, snappy, lz4, zstd). It additionally accepts none which is the default and equivalent to no compression.producer_linger_ms (integer, Minimum: 0, Maximum: 5000). This setting gives the upper bound on the delay for batching: once there is batch.size worth of records for a partition it will be sent immediately regardless of this setting, however if there are fewer than this many bytes accumulated for this partition the producer will linger for the specified time waiting for more records to show up. Defaults to 0.producer_max_request_size (integer, Minimum: 131072, Maximum: 67108864). This setting will limit the number of record batches the producer will send in a single request to avoid sending huge requests.scheduled_rebalance_max_delay_ms (integer, Minimum: 0, Maximum: 600000). The maximum delay that is scheduled in order to wait for the return of one or more departed workers before rebalancing and reassigning their connectors and tasks to the group. During this period the connectors and tasks of the departed workers remain unassigned. Defaults to 5 minutes.session_timeout_ms (integer, Minimum: 1, Maximum: 2147483647). The timeout in milliseconds used to detect failures when using Kafka\u2019s group management facilities (defaults to 10000).Appears on spec.userConfig.
Allow access to selected service ports from private networks.
Optional
kafka_connect (boolean). Allow clients to connect to kafka_connect with a DNS name that always resolves to the service's private IP addresses. Only available in certain network locations.prometheus (boolean). Allow clients to connect to prometheus with a DNS name that always resolves to the service's private IP addresses. Only available in certain network locations.Appears on spec.userConfig.
Allow access to selected service components through Privatelink.
Optional
jolokia (boolean). Enable jolokia.kafka_connect (boolean). Enable kafka_connect.prometheus (boolean). Enable prometheus.Appears on spec.userConfig.
Allow access to selected service ports from the public Internet.
Optional
kafka_connect (boolean). Allow clients to connect to kafka_connect from the public internet for service nodes that are in a project VPC or another type of private network.prometheus (boolean). Allow clients to connect to prometheus from the public internet for service nodes that are in a project VPC or another type of private network.Appears on spec.userConfig.
SecretProvider.
Required
name (string). Name of the secret provider. Used to reference secrets in connector config.Optional
aws (object). AWS config for Secret Provider. See below for nested schema.vault (object). Vault Config for Secret Provider. See below for nested schema.Appears on spec.userConfig.secret_providers.
AWS config for Secret Provider.
Required
auth_method (string, Enum: credentials). Auth method of the vault secret provider.region (string, MaxLength: 64). Region used to lookup secrets with AWS SecretManager.Optional
access_key (string, MaxLength: 128). Access key used to authenticate with aws.secret_key (string, MaxLength: 128). Secret key used to authenticate with aws.Appears on spec.userConfig.secret_providers.
Vault Config for Secret Provider.
Required
address (string, MinLength: 1, MaxLength: 65536). Address of the Vault server.auth_method (string, Enum: token). Auth method of the vault secret provider.Optional
engine_version (integer, Enum: 1, 2). KV Secrets Engine version of the Vault server instance.token (string, MaxLength: 256). Token used to authenticate with vault and auth method token.KafkaConnector is the Schema for the kafkaconnectors API.
Required
apiVersion (string). Value aiven.io/v1alpha1.kind (string). Value KafkaConnector.metadata (object). Data that identifies the object, including a name string and optional namespace.spec (object). KafkaConnectorSpec defines the desired state of KafkaConnector. See below for nested schema.Appears on KafkaConnector.
KafkaConnectorSpec defines the desired state of KafkaConnector.
Required
connectorClass (string, MaxLength: 1024). The Java class of the connector.project (string, Immutable, Pattern: ^[a-zA-Z0-9_-]+$, MaxLength: 63). Identifies the project this resource belongs to.serviceName (string, Immutable, Pattern: ^[a-z][-a-z0-9]+$, MaxLength: 63). Specifies the name of the service that this resource belongs to.userConfig (object, AdditionalProperties: string). The connector specific configuration To build config values from secret the template function {{ fromSecret \"name\" \"key\" }} is provided when interpreting the keys.Optional
authSecretRef (object). Authentication reference to Aiven token in a secret. See below for nested schema.Appears on spec.
Authentication reference to Aiven token in a secret.
Required
key (string, MinLength: 1).name (string, MinLength: 1).apiVersion: aiven.io/v1alpha1\nkind: KafkaSchema\nmetadata:\n name: my-schema\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n project: my-aiven-project\n serviceName: my-kafka\n subjectName: mny-subject\n compatibilityLevel: BACKWARD\n schema: |\n {\n \"doc\": \"example_doc\",\n \"fields\": [{\n \"default\": 5,\n \"doc\": \"field_doc\",\n \"name\": \"field_name\",\n \"namespace\": \"field_namespace\",\n \"type\": \"int\"\n }],\n \"name\": \"example_name\",\n \"namespace\": \"example_namespace\",\n \"type\": \"record\"\n }\nInfo
To create this resource, a Secret containing Aiven token must be created first.
Apply the resource with:
kubectl apply -f example.yaml\nVerify the newly created KafkaSchema:
kubectl get kafkaschemas my-schema\nThe output is similar to the following:
Name Service Name Project Subject Compatibility Level Version \nmy-schema my-kafka my-aiven-project mny-subject BACKWARD <version> \nKafkaSchema is the Schema for the kafkaschemas API.
Required
apiVersion (string). Value aiven.io/v1alpha1.kind (string). Value KafkaSchema.metadata (object). Data that identifies the object, including a name string and optional namespace.spec (object). KafkaSchemaSpec defines the desired state of KafkaSchema. See below for nested schema.Appears on KafkaSchema.
KafkaSchemaSpec defines the desired state of KafkaSchema.
Required
project (string, Immutable, Pattern: ^[a-zA-Z0-9_-]+$, MaxLength: 63). Identifies the project this resource belongs to.schema (string). Kafka Schema configuration should be a valid Avro Schema JSON format.serviceName (string, Immutable, Pattern: ^[a-z][-a-z0-9]+$, MaxLength: 63). Specifies the name of the service that this resource belongs to.subjectName (string, MaxLength: 63). Kafka Schema Subject name.Optional
authSecretRef (object). Authentication reference to Aiven token in a secret. See below for nested schema.compatibilityLevel (string, Enum: BACKWARD, BACKWARD_TRANSITIVE, FORWARD, FORWARD_TRANSITIVE, FULL, FULL_TRANSITIVE, NONE). Kafka Schemas compatibility level.Appears on spec.
Authentication reference to Aiven token in a secret.
Required
key (string, MinLength: 1).name (string, MinLength: 1).apiVersion: aiven.io/v1alpha1\nkind: KafkaSchemaRegistryACL\nmetadata:\n name: my-kafka-schema-registry-acl\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n project: aiven-project-name\n serviceName: my-kafka\n resource: Subject:my-topic\n username: my-user\n permission: schema_registry_read\nInfo
To create this resource, a Secret containing Aiven token must be created first.
Apply the resource with:
kubectl apply -f example.yaml\nVerify the newly created KafkaSchemaRegistryACL:
kubectl get kafkaschemaregistryacls my-kafka-schema-registry-acl\nThe output is similar to the following:
Name Project Service Name Resource Username State \nmy-kafka-schema-registry-acl aiven-project-name my-kafka Subject:my-topic my-user RUNNING \nKafkaSchemaRegistryACL is the Schema for the kafkaschemaregistryacls API.
Required
apiVersion (string). Value aiven.io/v1alpha1.kind (string). Value KafkaSchemaRegistryACL.metadata (object). Data that identifies the object, including a name string and optional namespace.spec (object). KafkaSchemaRegistryACLSpec defines the desired state of KafkaSchemaRegistryACL. See below for nested schema.Appears on KafkaSchemaRegistryACL.
KafkaSchemaRegistryACLSpec defines the desired state of KafkaSchemaRegistryACL.
Required
permission (string, Enum: schema_registry_read, schema_registry_write, Immutable).project (string, Immutable, Pattern: ^[a-zA-Z0-9_-]+$, MaxLength: 63). Identifies the project this resource belongs to.resource (string, Immutable, MaxLength: 249). Resource name pattern for the Schema Registry ACL entry.serviceName (string, Immutable, Pattern: ^[a-z][-a-z0-9]+$, MaxLength: 63). Specifies the name of the service that this resource belongs to.username (string, Immutable, MaxLength: 64). Username pattern for the ACL entry.Optional
authSecretRef (object). Authentication reference to Aiven token in a secret. See below for nested schema.Appears on spec.
Authentication reference to Aiven token in a secret.
Required
key (string, MinLength: 1).name (string, MinLength: 1).apiVersion: aiven.io/v1alpha1\nkind: KafkaTopic\nmetadata:\n name: kafka-topic\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n project: my-aiven-project\n serviceName: my-kafka\n topicName: my-kafka-topic\n\n replication: 2\n partitions: 1\n\n config:\n min_cleanable_dirty_ratio: 0.2\nInfo
To create this resource, a Secret containing Aiven token must be created first.
Apply the resource with:
kubectl apply -f example.yaml\nVerify the newly created KafkaTopic:
kubectl get kafkatopics kafka-topic\nThe output is similar to the following:
Name Service Name Project Partitions Replication \nkafka-topic my-kafka my-aiven-project 1 2 \nKafkaTopic is the Schema for the kafkatopics API.
Required
apiVersion (string). Value aiven.io/v1alpha1.kind (string). Value KafkaTopic.metadata (object). Data that identifies the object, including a name string and optional namespace.spec (object). KafkaTopicSpec defines the desired state of KafkaTopic. See below for nested schema.Appears on KafkaTopic.
KafkaTopicSpec defines the desired state of KafkaTopic.
Required
partitions (integer, Minimum: 1, Maximum: 1000000). Number of partitions to create in the topic.project (string, Immutable, Pattern: ^[a-zA-Z0-9_-]+$, MaxLength: 63). Identifies the project this resource belongs to.replication (integer, Minimum: 2). Replication factor for the topic.serviceName (string, Immutable, Pattern: ^[a-z][-a-z0-9]+$, MaxLength: 63). Specifies the name of the service that this resource belongs to.Optional
authSecretRef (object). Authentication reference to Aiven token in a secret. See below for nested schema.config (object). Kafka topic configuration. See below for nested schema.tags (array of objects). Kafka topic tags. See below for nested schema.termination_protection (boolean). It is a Kubernetes side deletion protections, which prevents the kafka topic from being deleted by Kubernetes. It is recommended to enable this for any production databases containing critical data.topicName (string, Immutable, MinLength: 1, MaxLength: 249). Topic name. If provided, is used instead of metadata.name. This field supports additional characters, has a longer length, and will replace metadata.name in future releases.Appears on spec.
Authentication reference to Aiven token in a secret.
Required
key (string, MinLength: 1).name (string, MinLength: 1).Appears on spec.
Kafka topic configuration.
Optional
cleanup_policy (string). cleanup.policy value.compression_type (string). compression.type value.delete_retention_ms (integer). delete.retention.ms value.file_delete_delay_ms (integer). file.delete.delay.ms value.flush_messages (integer). flush.messages value.flush_ms (integer). flush.ms value.index_interval_bytes (integer). index.interval.bytes value.local_retention_bytes (integer). local.retention.bytes value.local_retention_ms (integer). local.retention.ms value.max_compaction_lag_ms (integer). max.compaction.lag.ms value.max_message_bytes (integer). max.message.bytes value.message_downconversion_enable (boolean). message.downconversion.enable value.message_format_version (string). message.format.version value.message_timestamp_difference_max_ms (integer). message.timestamp.difference.max.ms value.message_timestamp_type (string). message.timestamp.type value.min_cleanable_dirty_ratio (number). min.cleanable.dirty.ratio value.min_compaction_lag_ms (integer). min.compaction.lag.ms value.min_insync_replicas (integer). min.insync.replicas value.preallocate (boolean). preallocate value.remote_storage_enable (boolean). remote_storage_enable.retention_bytes (integer). retention.bytes value.retention_ms (integer). retention.ms value.segment_bytes (integer). segment.bytes value.segment_index_bytes (integer). segment.index.bytes value.segment_jitter_ms (integer). segment.jitter.ms value.segment_ms (integer). segment.ms value.Appears on spec.
Kafka topic tags.
Required
key (string, Pattern: ^[a-zA-Z0-9_-]+$, MinLength: 1, MaxLength: 64).Optional
value (string, Pattern: ^[a-zA-Z0-9_-]+$, MaxLength: 256).apiVersion: aiven.io/v1alpha1\nkind: MySQL\nmetadata:\n name: my-mysql\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n connInfoSecretTarget:\n name: mysql-secret\n prefix: MY_SECRET_PREFIX_\n annotations:\n foo: bar\n labels:\n baz: egg\n\n project: my-aiven-project\n cloudName: google-europe-west1\n plan: business-4\n\n maintenanceWindowDow: sunday\n maintenanceWindowTime: 11:00:00\n\n userConfig:\n backup_hour: 17\n backup_minute: 11\n ip_filter:\n - network: 0.0.0.0\n description: whatever\n - network: 10.20.0.0/16\nInfo
To create this resource, a Secret containing Aiven token must be created first.
Apply the resource with:
kubectl apply -f example.yaml\nVerify the newly created MySQL:
kubectl get mysqls my-mysql\nThe output is similar to the following:
Name Project Region Plan State \nmy-mysql my-aiven-project google-europe-west1 business-4 RUNNING \nTo view the details of the Secret, use the following command:
kubectl describe secret mysql-secret\nYou can use the jq to quickly decode the Secret:
kubectl get secret mysql-secret -o json | jq '.data | map_values(@base64d)'\nThe output is similar to the following:
{\n \"MYSQL_HOST\": \"<secret>\",\n \"MYSQL_PORT\": \"<secret>\",\n \"MYSQL_DATABASE\": \"<secret>\",\n \"MYSQL_USER\": \"<secret>\",\n \"MYSQL_PASSWORD\": \"<secret>\",\n \"MYSQL_SSL_MODE\": \"<secret>\",\n \"MYSQL_URI\": \"<secret>\",\n \"MYSQL_REPLICA_URI\": \"<secret>\",\n \"MYSQL_CA_CERT\": \"<secret>\",\n}\nMySQL is the Schema for the mysqls API.
Exposes secret keys
MYSQL_HOST, MYSQL_PORT, MYSQL_DATABASE, MYSQL_USER, MYSQL_PASSWORD, MYSQL_SSL_MODE, MYSQL_URI, MYSQL_REPLICA_URI, MYSQL_CA_CERT.
Required
apiVersion (string). Value aiven.io/v1alpha1.kind (string). Value MySQL.metadata (object). Data that identifies the object, including a name string and optional namespace.spec (object). MySQLSpec defines the desired state of MySQL. See below for nested schema.Appears on MySQL.
MySQLSpec defines the desired state of MySQL.
Required
plan (string, MaxLength: 128). Subscription plan.project (string, Immutable, Pattern: ^[a-zA-Z0-9_-]+$, MaxLength: 63). Identifies the project this resource belongs to.Optional
authSecretRef (object). Authentication reference to Aiven token in a secret. See below for nested schema.cloudName (string, MaxLength: 256). Cloud the service runs in.connInfoSecretTarget (object). Secret configuration. See below for nested schema.connInfoSecretTargetDisabled (boolean, Immutable). When true, the secret containing connection information will not be created, defaults to false. This field cannot be changed after resource creation.disk_space (string, Pattern: (?i)^[1-9][0-9]*(GiB|G)?$). The disk space of the service, possible values depend on the service type, the cloud provider and the project. Reducing will result in the service re-balancing. The removal of this field does not change the value.maintenanceWindowDow (string, Enum: monday, tuesday, wednesday, thursday, friday, saturday, sunday). Day of week when maintenance operations should be performed. One monday, tuesday, wednesday, etc.maintenanceWindowTime (string, MaxLength: 8). Time of day when maintenance operations should be performed. UTC time in HH:mm:ss format.projectVPCRef (object). ProjectVPCRef reference to ProjectVPC resource to use its ID as ProjectVPCID automatically. See below for nested schema.projectVpcId (string, MaxLength: 36). Identifier of the VPC the service should be in, if any.serviceIntegrations (array of objects, Immutable, MaxItems: 1). Service integrations to specify when creating a service. Not applied after initial service creation. See below for nested schema.tags (object, AdditionalProperties: string). Tags are key-value pairs that allow you to categorize services.technicalEmails (array of objects, MaxItems: 10). Defines the email addresses that will receive alerts about upcoming maintenance updates or warnings about service instability. See below for nested schema.terminationProtection (boolean). Prevent service from being deleted. It is recommended to have this enabled for all services.userConfig (object). MySQL specific user configuration options. See below for nested schema.Appears on spec.
Authentication reference to Aiven token in a secret.
Required
key (string, MinLength: 1).name (string, MinLength: 1).Appears on spec.
Secret configuration.
Required
name (string, Immutable). Name of the secret resource to be created. By default, it is equal to the resource name.Optional
annotations (object, AdditionalProperties: string). Annotations added to the secret.labels (object, AdditionalProperties: string). Labels added to the secret.prefix (string). Prefix for the secret's keys. Added \"as is\" without any transformations. By default, is equal to the kind name in uppercase + underscore, e.g. KAFKA_, REDIS_, etc.Appears on spec.
ProjectVPCRef reference to ProjectVPC resource to use its ID as ProjectVPCID automatically.
Required
name (string, MinLength: 1).Optional
namespace (string, MinLength: 1).Appears on spec.
Service integrations to specify when creating a service. Not applied after initial service creation.
Required
integrationType (string, Enum: read_replica).sourceServiceName (string, MinLength: 1, MaxLength: 64).Appears on spec.
Defines the email addresses that will receive alerts about upcoming maintenance updates or warnings about service instability.
Required
email (string). Email address.Appears on spec.
MySQL specific user configuration options.
Optional
additional_backup_regions (array of strings, MaxItems: 1). Additional Cloud Regions for Backup Replication.admin_password (string, Immutable, Pattern: ^[a-zA-Z0-9-_]+$, MinLength: 8, MaxLength: 256). Custom password for admin user. Defaults to random string. This must be set only when a new service is being created.admin_username (string, Immutable, Pattern: ^[_A-Za-z0-9][-._A-Za-z0-9]{0,63}$, MaxLength: 64). Custom username for admin user. This must be set only when a new service is being created.backup_hour (integer, Minimum: 0, Maximum: 23). The hour of day (in UTC) when backup for the service is started. New backup is only started if previous backup has already completed.backup_minute (integer, Minimum: 0, Maximum: 59). The minute of an hour when backup for the service is started. New backup is only started if previous backup has already completed.binlog_retention_period (integer, Minimum: 600, Maximum: 86400). The minimum amount of time in seconds to keep binlog entries before deletion. This may be extended for services that require binlog entries for longer than the default for example if using the MySQL Debezium Kafka connector.ip_filter (array of objects, MaxItems: 1024). Allow incoming connections from CIDR address block, e.g. 10.20.0.0/16. See below for nested schema.migration (object). Migrate data from existing server. See below for nested schema.mysql (object). mysql.conf configuration values. See below for nested schema.mysql_version (string, Enum: 8). MySQL major version.private_access (object). Allow access to selected service ports from private networks. See below for nested schema.privatelink_access (object). Allow access to selected service components through Privatelink. See below for nested schema.project_to_fork_from (string, Immutable, Pattern: ^[a-z][-a-z0-9]{0,63}$|^$, MaxLength: 63). Name of another project to fork a service from. This has effect only when a new service is being created.public_access (object). Allow access to selected service ports from the public Internet. See below for nested schema.recovery_target_time (string, Immutable, MaxLength: 32). Recovery target time when forking a service. This has effect only when a new service is being created.service_log (boolean). Store logs for the service so that they are available in the HTTP API and console.service_to_fork_from (string, Immutable, Pattern: ^[a-z][-a-z0-9]{0,63}$|^$, MaxLength: 64). Name of another service to fork from. This has effect only when a new service is being created.static_ips (boolean). Use static public IP addresses.Appears on spec.userConfig.
CIDR address block, either as a string, or in a dict with an optional description field.
Required
network (string, MaxLength: 43). CIDR address block.Optional
description (string, MaxLength: 1024). Description for IP filter list entry.Appears on spec.userConfig.
Migrate data from existing server.
Required
host (string, MaxLength: 255). Hostname or IP address of the server where to migrate data from.port (integer, Minimum: 1, Maximum: 65535). Port number of the server where to migrate data from.Optional
dbname (string, MaxLength: 63). Database name for bootstrapping the initial connection.ignore_dbs (string, MaxLength: 2048). Comma-separated list of databases, which should be ignored during migration (supported by MySQL and PostgreSQL only at the moment).method (string, Enum: dump, replication). The migration method to be used (currently supported only by Redis, Dragonfly, MySQL and PostgreSQL service types).password (string, MaxLength: 256). Password for authentication with the server where to migrate data from.ssl (boolean). The server where to migrate data from is secured with SSL.username (string, MaxLength: 256). User name for authentication with the server where to migrate data from.Appears on spec.userConfig.
mysql.conf configuration values.
Optional
connect_timeout (integer, Minimum: 2, Maximum: 3600). The number of seconds that the mysqld server waits for a connect packet before responding with Bad handshake.default_time_zone (string, Pattern: ^([-+][\\d:]*|[\\w/]*)$, MinLength: 2, MaxLength: 100). Default server time zone as an offset from UTC (from -12:00 to +12:00), a time zone name, or SYSTEM to use the MySQL server default.group_concat_max_len (integer, Minimum: 4). The maximum permitted result length in bytes for the GROUP_CONCAT() function.information_schema_stats_expiry (integer, Minimum: 900, Maximum: 31536000). The time, in seconds, before cached statistics expire.innodb_change_buffer_max_size (integer, Minimum: 0, Maximum: 50). Maximum size for the InnoDB change buffer, as a percentage of the total size of the buffer pool. Default is 25.innodb_flush_neighbors (integer, Minimum: 0, Maximum: 2). Specifies whether flushing a page from the InnoDB buffer pool also flushes other dirty pages in the same extent (default is 1): 0 - dirty pages in the same extent are not flushed, 1 - flush contiguous dirty pages in the same extent, 2 - flush dirty pages in the same extent.innodb_ft_min_token_size (integer, Minimum: 0, Maximum: 16). Minimum length of words that are stored in an InnoDB FULLTEXT index. Changing this parameter will lead to a restart of the MySQL service.innodb_ft_server_stopword_table (string, Pattern: ^.+/.+$, MaxLength: 1024). This option is used to specify your own InnoDB FULLTEXT index stopword list for all InnoDB tables.innodb_lock_wait_timeout (integer, Minimum: 1, Maximum: 3600). The length of time in seconds an InnoDB transaction waits for a row lock before giving up. Default is 120.innodb_log_buffer_size (integer, Minimum: 1048576, Maximum: 4294967295). The size in bytes of the buffer that InnoDB uses to write to the log files on disk.innodb_online_alter_log_max_size (integer, Minimum: 65536, Maximum: 1099511627776). The upper limit in bytes on the size of the temporary log files used during online DDL operations for InnoDB tables.innodb_print_all_deadlocks (boolean). When enabled, information about all deadlocks in InnoDB user transactions is recorded in the error log. Disabled by default.innodb_read_io_threads (integer, Minimum: 1, Maximum: 64). The number of I/O threads for read operations in InnoDB. Default is 4. Changing this parameter will lead to a restart of the MySQL service.innodb_rollback_on_timeout (boolean). When enabled a transaction timeout causes InnoDB to abort and roll back the entire transaction. Changing this parameter will lead to a restart of the MySQL service.innodb_thread_concurrency (integer, Minimum: 0, Maximum: 1000). Defines the maximum number of threads permitted inside of InnoDB. Default is 0 (infinite concurrency - no limit).innodb_write_io_threads (integer, Minimum: 1, Maximum: 64). The number of I/O threads for write operations in InnoDB. Default is 4. Changing this parameter will lead to a restart of the MySQL service.interactive_timeout (integer, Minimum: 30, Maximum: 604800). The number of seconds the server waits for activity on an interactive connection before closing it.internal_tmp_mem_storage_engine (string, Enum: TempTable, MEMORY). The storage engine for in-memory internal temporary tables.long_query_time (number, Minimum: 0, Maximum: 3600). The slow_query_logs work as SQL statements that take more than long_query_time seconds to execute. Default is 10s.max_allowed_packet (integer, Minimum: 102400, Maximum: 1073741824). Size of the largest message in bytes that can be received by the server. Default is 67108864 (64M).max_heap_table_size (integer, Minimum: 1048576, Maximum: 1073741824). Limits the size of internal in-memory tables. Also set tmp_table_size. Default is 16777216 (16M).net_buffer_length (integer, Minimum: 1024, Maximum: 1048576). Start sizes of connection buffer and result buffer. Default is 16384 (16K). Changing this parameter will lead to a restart of the MySQL service.net_read_timeout (integer, Minimum: 1, Maximum: 3600). The number of seconds to wait for more data from a connection before aborting the read.net_write_timeout (integer, Minimum: 1, Maximum: 3600). The number of seconds to wait for a block to be written to a connection before aborting the write.slow_query_log (boolean). Slow query log enables capturing of slow queries. Setting slow_query_log to false also truncates the mysql.slow_log table. Default is off.sort_buffer_size (integer, Minimum: 32768, Maximum: 1073741824). Sort buffer size in bytes for ORDER BY optimization. Default is 262144 (256K).sql_mode (string, Pattern: ^[A-Z_]*(,[A-Z_]+)*$, MaxLength: 1024). Global SQL mode. Set to empty to use MySQL server defaults. When creating a new service and not setting this field Aiven default SQL mode (strict, SQL standard compliant) will be assigned.sql_require_primary_key (boolean). Require primary key to be defined for new tables or old tables modified with ALTER TABLE and fail if missing. It is recommended to always have primary keys because various functionality may break if any large table is missing them.tmp_table_size (integer, Minimum: 1048576, Maximum: 1073741824). Limits the size of internal in-memory tables. Also set max_heap_table_size. Default is 16777216 (16M).wait_timeout (integer, Minimum: 1, Maximum: 2147483). The number of seconds the server waits for activity on a noninteractive connection before closing it.Appears on spec.userConfig.
Allow access to selected service ports from private networks.
Optional
mysql (boolean). Allow clients to connect to mysql with a DNS name that always resolves to the service's private IP addresses. Only available in certain network locations.mysqlx (boolean). Allow clients to connect to mysqlx with a DNS name that always resolves to the service's private IP addresses. Only available in certain network locations.prometheus (boolean). Allow clients to connect to prometheus with a DNS name that always resolves to the service's private IP addresses. Only available in certain network locations.Appears on spec.userConfig.
Allow access to selected service components through Privatelink.
Optional
mysql (boolean). Enable mysql.mysqlx (boolean). Enable mysqlx.prometheus (boolean). Enable prometheus.Appears on spec.userConfig.
Allow access to selected service ports from the public Internet.
Optional
mysql (boolean). Allow clients to connect to mysql from the public internet for service nodes that are in a project VPC or another type of private network.mysqlx (boolean). Allow clients to connect to mysqlx from the public internet for service nodes that are in a project VPC or another type of private network.prometheus (boolean). Allow clients to connect to prometheus from the public internet for service nodes that are in a project VPC or another type of private network.apiVersion: aiven.io/v1alpha1\nkind: OpenSearch\nmetadata:\n name: my-os\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n connInfoSecretTarget:\n name: os-secret\n prefix: MY_SECRET_PREFIX_\n annotations:\n foo: bar\n labels:\n baz: egg\n\n project: my-aiven-project\n cloudName: google-europe-west1\n plan: startup-4\n disk_space: 80GiB\n\n maintenanceWindowDow: friday\n maintenanceWindowTime: 23:00:00\nInfo
To create this resource, a Secret containing Aiven token must be created first.
Apply the resource with:
kubectl apply -f example.yaml\nVerify the newly created OpenSearch:
kubectl get opensearches my-os\nThe output is similar to the following:
Name Project Region Plan State \nmy-os my-aiven-project google-europe-west1 startup-4 RUNNING \nTo view the details of the Secret, use the following command:
kubectl describe secret os-secret\nYou can use the jq to quickly decode the Secret:
kubectl get secret os-secret -o json | jq '.data | map_values(@base64d)'\nThe output is similar to the following:
{\n \"OPENSEARCH_HOST\": \"<secret>\",\n \"OPENSEARCH_PORT\": \"<secret>\",\n \"OPENSEARCH_USER\": \"<secret>\",\n \"OPENSEARCH_PASSWORD\": \"<secret>\",\n}\nOpenSearch is the Schema for the opensearches API.
Exposes secret keys
OPENSEARCH_HOST, OPENSEARCH_PORT, OPENSEARCH_USER, OPENSEARCH_PASSWORD.
Required
apiVersion (string). Value aiven.io/v1alpha1.kind (string). Value OpenSearch.metadata (object). Data that identifies the object, including a name string and optional namespace.spec (object). OpenSearchSpec defines the desired state of OpenSearch. See below for nested schema.Appears on OpenSearch.
OpenSearchSpec defines the desired state of OpenSearch.
Required
plan (string, MaxLength: 128). Subscription plan.project (string, Immutable, Pattern: ^[a-zA-Z0-9_-]+$, MaxLength: 63). Identifies the project this resource belongs to.Optional
authSecretRef (object). Authentication reference to Aiven token in a secret. See below for nested schema.cloudName (string, MaxLength: 256). Cloud the service runs in.connInfoSecretTarget (object). Secret configuration. See below for nested schema.connInfoSecretTargetDisabled (boolean, Immutable). When true, the secret containing connection information will not be created, defaults to false. This field cannot be changed after resource creation.disk_space (string, Pattern: (?i)^[1-9][0-9]*(GiB|G)?$). The disk space of the service, possible values depend on the service type, the cloud provider and the project. Reducing will result in the service re-balancing. The removal of this field does not change the value.maintenanceWindowDow (string, Enum: monday, tuesday, wednesday, thursday, friday, saturday, sunday). Day of week when maintenance operations should be performed. One monday, tuesday, wednesday, etc.maintenanceWindowTime (string, MaxLength: 8). Time of day when maintenance operations should be performed. UTC time in HH:mm:ss format.projectVPCRef (object). ProjectVPCRef reference to ProjectVPC resource to use its ID as ProjectVPCID automatically. See below for nested schema.projectVpcId (string, MaxLength: 36). Identifier of the VPC the service should be in, if any.serviceIntegrations (array of objects, Immutable, MaxItems: 1). Service integrations to specify when creating a service. Not applied after initial service creation. See below for nested schema.tags (object, AdditionalProperties: string). Tags are key-value pairs that allow you to categorize services.technicalEmails (array of objects, MaxItems: 10). Defines the email addresses that will receive alerts about upcoming maintenance updates or warnings about service instability. See below for nested schema.terminationProtection (boolean). Prevent service from being deleted. It is recommended to have this enabled for all services.userConfig (object). OpenSearch specific user configuration options. See below for nested schema.Appears on spec.
Authentication reference to Aiven token in a secret.
Required
key (string, MinLength: 1).name (string, MinLength: 1).Appears on spec.
Secret configuration.
Required
name (string, Immutable). Name of the secret resource to be created. By default, it is equal to the resource name.Optional
annotations (object, AdditionalProperties: string). Annotations added to the secret.labels (object, AdditionalProperties: string). Labels added to the secret.prefix (string). Prefix for the secret's keys. Added \"as is\" without any transformations. By default, is equal to the kind name in uppercase + underscore, e.g. KAFKA_, REDIS_, etc.Appears on spec.
ProjectVPCRef reference to ProjectVPC resource to use its ID as ProjectVPCID automatically.
Required
name (string, MinLength: 1).Optional
namespace (string, MinLength: 1).Appears on spec.
Service integrations to specify when creating a service. Not applied after initial service creation.
Required
integrationType (string, Enum: read_replica).sourceServiceName (string, MinLength: 1, MaxLength: 64).Appears on spec.
Defines the email addresses that will receive alerts about upcoming maintenance updates or warnings about service instability.
Required
email (string). Email address.Appears on spec.
OpenSearch specific user configuration options.
Optional
additional_backup_regions (array of strings, MaxItems: 1). Additional Cloud Regions for Backup Replication.custom_domain (string, MaxLength: 255). Serve the web frontend using a custom CNAME pointing to the Aiven DNS name.disable_replication_factor_adjustment (boolean). DEPRECATED: Disable automatic replication factor adjustment for multi-node services. By default, Aiven ensures all indexes are replicated at least to two nodes. Note: Due to potential data loss in case of losing a service node, this setting can no longer be activated.index_patterns (array of objects, MaxItems: 512). Index patterns. See below for nested schema.index_template (object). Template settings for all new indexes. See below for nested schema.ip_filter (array of objects, MaxItems: 1024). Allow incoming connections from CIDR address block, e.g. 10.20.0.0/16. See below for nested schema.keep_index_refresh_interval (boolean). Aiven automation resets index.refresh_interval to default value for every index to be sure that indices are always visible to search. If it doesn't fit your case, you can disable this by setting up this flag to true.max_index_count (integer, Minimum: 0). DEPRECATED: use index_patterns instead.openid (object). OpenSearch OpenID Connect Configuration. See below for nested schema.opensearch (object). OpenSearch settings. See below for nested schema.opensearch_dashboards (object). OpenSearch Dashboards settings. See below for nested schema.opensearch_version (string, Enum: 1, 2). OpenSearch major version.private_access (object). Allow access to selected service ports from private networks. See below for nested schema.privatelink_access (object). Allow access to selected service components through Privatelink. See below for nested schema.project_to_fork_from (string, Immutable, Pattern: ^[a-z][-a-z0-9]{0,63}$|^$, MaxLength: 63). Name of another project to fork a service from. This has effect only when a new service is being created.public_access (object). Allow access to selected service ports from the public Internet. See below for nested schema.recovery_basebackup_name (string, Pattern: ^[a-zA-Z0-9-_:.]+$, MaxLength: 128). Name of the basebackup to restore in forked service.saml (object). OpenSearch SAML configuration. See below for nested schema.service_log (boolean). Store logs for the service so that they are available in the HTTP API and console.service_to_fork_from (string, Immutable, Pattern: ^[a-z][-a-z0-9]{0,63}$|^$, MaxLength: 64). Name of another service to fork from. This has effect only when a new service is being created.static_ips (boolean). Use static public IP addresses.Appears on spec.userConfig.
Allows you to create glob style patterns and set a max number of indexes matching this pattern you want to keep. Creating indexes exceeding this value will cause the oldest one to get deleted. You could for example create a pattern looking like logs.? and then create index logs.1, logs.2 etc, it will delete logs.1 once you create logs.6. Do note logs.? does not apply to logs.10. Note: Setting max_index_count to 0 will do nothing and the pattern gets ignored.
Required
max_index_count (integer, Minimum: 0). Maximum number of indexes to keep.pattern (string, Pattern: ^[A-Za-z0-9-_.*?]+$, MaxLength: 1024). fnmatch pattern.Optional
sorting_algorithm (string, Enum: alphabetical, creation_date). Deletion sorting algorithm.Appears on spec.userConfig.
Template settings for all new indexes.
Optional
mapping_nested_objects_limit (integer, Minimum: 0, Maximum: 100000). The maximum number of nested JSON objects that a single document can contain across all nested types. This limit helps to prevent out of memory errors when a document contains too many nested objects. Default is 10000.number_of_replicas (integer, Minimum: 0, Maximum: 29). The number of replicas each primary shard has.number_of_shards (integer, Minimum: 1, Maximum: 1024). The number of primary shards that an index should have.Appears on spec.userConfig.
CIDR address block, either as a string, or in a dict with an optional description field.
Required
network (string, MaxLength: 43). CIDR address block.Optional
description (string, MaxLength: 1024). Description for IP filter list entry.Appears on spec.userConfig.
OpenSearch OpenID Connect Configuration.
Required
client_id (string, Pattern: ^[^\\r\\n]*$, MinLength: 1, MaxLength: 1024). The ID of the OpenID Connect client configured in your IdP. Required.client_secret (string, Pattern: ^[^\\r\\n]*$, MinLength: 1, MaxLength: 1024). The client secret of the OpenID Connect client configured in your IdP. Required.connect_url (string, MaxLength: 2048). The URL of your IdP where the Security plugin can find the OpenID Connect metadata/configuration settings.enabled (boolean). Enables or disables OpenID Connect authentication for OpenSearch. When enabled, users can authenticate using OpenID Connect with an Identity Provider.Optional
header (string, Pattern: ^[^\\r\\n]*$, MinLength: 1, MaxLength: 1024). HTTP header name of the JWT token. Optional. Default is Authorization.jwt_header (string, Pattern: ^[^\\r\\n]*$, MinLength: 1, MaxLength: 1024). The HTTP header that stores the token. Typically the Authorization header with the Bearer schema: Authorization: Bearer . Optional. Default is Authorization. jwt_url_parameter (string, Pattern: ^[^\\r\\n]*$, MinLength: 1, MaxLength: 1024). If the token is not transmitted in the HTTP header, but as an URL parameter, define the name of the parameter here. Optional.refresh_rate_limit_count (integer, Minimum: 10). The maximum number of unknown key IDs in the time frame. Default is 10. Optional.refresh_rate_limit_time_window_ms (integer, Minimum: 10000). The time frame to use when checking the maximum number of unknown key IDs, in milliseconds. Optional.Default is 10000 (10 seconds).roles_key (string, Pattern: ^[^\\r\\n]*$, MinLength: 1, MaxLength: 1024). The key in the JSON payload that stores the user\u2019s roles. The value of this key must be a comma-separated list of roles. Required only if you want to use roles in the JWT.scope (string, Pattern: ^[^\\r\\n]*$, MinLength: 1, MaxLength: 1024). The scope of the identity token issued by the IdP. Optional. Default is openid profile email address phone.subject_key (string, Pattern: ^[^\\r\\n]*$, MinLength: 1, MaxLength: 1024). The key in the JSON payload that stores the user\u2019s name. If not defined, the subject registered claim is used. Most IdP providers use the preferred_username claim. Optional.Appears on spec.userConfig.
OpenSearch settings.
Optional
action_auto_create_index_enabled (boolean). Explicitly allow or block automatic creation of indices. Defaults to true.action_destructive_requires_name (boolean). Require explicit index names when deleting.auth_failure_listeners (object). Opensearch Security Plugin Settings. See below for nested schema.cluster_max_shards_per_node (integer, Minimum: 100, Maximum: 10000). Controls the number of shards allowed in the cluster per data node.cluster_routing_allocation_node_concurrent_recoveries (integer, Minimum: 2, Maximum: 16). How many concurrent incoming/outgoing shard recoveries (normally replicas) are allowed to happen on a node. Defaults to 2.email_sender_name (string, Pattern: ^[a-zA-Z0-9-_]+$, MaxLength: 40). Sender name placeholder to be used in Opensearch Dashboards and Opensearch keystore.email_sender_password (string, Pattern: ^[^\\x00-\\x1F]+$, MaxLength: 1024). Sender password for Opensearch alerts to authenticate with SMTP server.email_sender_username (string, Pattern: ^[^\\x00-\\x1F]+$, MaxLength: 320). Sender username for Opensearch alerts.enable_security_audit (boolean). Enable/Disable security audit.http_max_content_length (integer, Minimum: 1, Maximum: 2147483647). Maximum content length for HTTP requests to the OpenSearch HTTP API, in bytes.http_max_header_size (integer, Minimum: 1024, Maximum: 262144). The max size of allowed headers, in bytes.http_max_initial_line_length (integer, Minimum: 1024, Maximum: 65536). The max length of an HTTP URL, in bytes.indices_fielddata_cache_size (integer, Minimum: 3, Maximum: 100). Relative amount. Maximum amount of heap memory used for field data cache. This is an expert setting; decreasing the value too much will increase overhead of loading field data; too much memory used for field data cache will decrease amount of heap available for other operations.indices_memory_index_buffer_size (integer, Minimum: 3, Maximum: 40). Percentage value. Default is 10%. Total amount of heap used for indexing buffer, before writing segments to disk. This is an expert setting. Too low value will slow down indexing; too high value will increase indexing performance but causes performance issues for query performance.indices_memory_max_index_buffer_size (integer, Minimum: 3, Maximum: 2048). Absolute value. Default is unbound. Doesn't work without indices.memory.index_buffer_size. Maximum amount of heap used for query cache, an absolute indices.memory.index_buffer_size maximum hard limit.indices_memory_min_index_buffer_size (integer, Minimum: 3, Maximum: 2048). Absolute value. Default is 48mb. Doesn't work without indices.memory.index_buffer_size. Minimum amount of heap used for query cache, an absolute indices.memory.index_buffer_size minimal hard limit.indices_queries_cache_size (integer, Minimum: 3, Maximum: 40). Percentage value. Default is 10%. Maximum amount of heap used for query cache. This is an expert setting. Too low value will decrease query performance and increase performance for other operations; too high value will cause issues with other OpenSearch functionality.indices_query_bool_max_clause_count (integer, Minimum: 64, Maximum: 4096). Maximum number of clauses Lucene BooleanQuery can have. The default value (1024) is relatively high, and increasing it may cause performance issues. Investigate other approaches first before increasing this value.indices_recovery_max_bytes_per_sec (integer, Minimum: 40, Maximum: 400). Limits total inbound and outbound recovery traffic for each node. Applies to both peer recoveries as well as snapshot recoveries (i.e., restores from a snapshot). Defaults to 40mb.indices_recovery_max_concurrent_file_chunks (integer, Minimum: 2, Maximum: 5). Number of file chunks sent in parallel for each recovery. Defaults to 2.ism_enabled (boolean). Specifies whether ISM is enabled or not.ism_history_enabled (boolean). Specifies whether audit history is enabled or not. The logs from ISM are automatically indexed to a logs document.ism_history_max_age (integer, Minimum: 1, Maximum: 2147483647). The maximum age before rolling over the audit history index in hours.ism_history_max_docs (integer, Minimum: 1). The maximum number of documents before rolling over the audit history index.ism_history_rollover_check_period (integer, Minimum: 1, Maximum: 2147483647). The time between rollover checks for the audit history index in hours.ism_history_rollover_retention_period (integer, Minimum: 1, Maximum: 2147483647). How long audit history indices are kept in days.knn_memory_circuit_breaker_enabled (boolean). Enable or disable KNN memory circuit breaker. Defaults to true.knn_memory_circuit_breaker_limit (integer, Minimum: 3, Maximum: 100). Maximum amount of memory that can be used for KNN index. Defaults to 50% of the JVM heap size.override_main_response_version (boolean). Compatibility mode sets OpenSearch to report its version as 7.10 so clients continue to work. Default is false.plugins_alerting_filter_by_backend_roles (boolean). Enable or disable filtering of alerting by backend roles. Requires Security plugin. Defaults to false.reindex_remote_whitelist (array of strings, MaxItems: 32). Whitelisted addresses for reindexing. Changing this value will cause all OpenSearch instances to restart.script_max_compilations_rate (string, MaxLength: 1024). Script compilation circuit breaker limits the number of inline script compilations within a period of time. Default is use-context.search_max_buckets (integer, Minimum: 1, Maximum: 1000000). Maximum number of aggregation buckets allowed in a single response. OpenSearch default value is used when this is not defined.thread_pool_analyze_queue_size (integer, Minimum: 10, Maximum: 2000). Size for the thread pool queue. See documentation for exact details.thread_pool_analyze_size (integer, Minimum: 1, Maximum: 128). Size for the thread pool. See documentation for exact details. Do note this may have maximum value depending on CPU count - value is automatically lowered if set to higher than maximum value.thread_pool_force_merge_size (integer, Minimum: 1, Maximum: 128). Size for the thread pool. See documentation for exact details. Do note this may have maximum value depending on CPU count - value is automatically lowered if set to higher than maximum value.thread_pool_get_queue_size (integer, Minimum: 10, Maximum: 2000). Size for the thread pool queue. See documentation for exact details.thread_pool_get_size (integer, Minimum: 1, Maximum: 128). Size for the thread pool. See documentation for exact details. Do note this may have maximum value depending on CPU count - value is automatically lowered if set to higher than maximum value.thread_pool_search_queue_size (integer, Minimum: 10, Maximum: 2000). Size for the thread pool queue. See documentation for exact details.thread_pool_search_size (integer, Minimum: 1, Maximum: 128). Size for the thread pool. See documentation for exact details. Do note this may have maximum value depending on CPU count - value is automatically lowered if set to higher than maximum value.thread_pool_search_throttled_queue_size (integer, Minimum: 10, Maximum: 2000). Size for the thread pool queue. See documentation for exact details.thread_pool_search_throttled_size (integer, Minimum: 1, Maximum: 128). Size for the thread pool. See documentation for exact details. Do note this may have maximum value depending on CPU count - value is automatically lowered if set to higher than maximum value.thread_pool_write_queue_size (integer, Minimum: 10, Maximum: 2000). Size for the thread pool queue. See documentation for exact details.thread_pool_write_size (integer, Minimum: 1, Maximum: 128). Size for the thread pool. See documentation for exact details. Do note this may have maximum value depending on CPU count - value is automatically lowered if set to higher than maximum value.Appears on spec.userConfig.opensearch.
Opensearch Security Plugin Settings.
Optional
internal_authentication_backend_limiting (object). See below for nested schema.ip_rate_limiting (object). IP address rate limiting settings. See below for nested schema.Appears on spec.userConfig.opensearch.auth_failure_listeners.
Optional
allowed_tries (integer, Minimum: 0, Maximum: 2147483647). The number of login attempts allowed before login is blocked.authentication_backend (string, Enum: internal, MaxLength: 1024). internal_authentication_backend_limiting.authentication_backend.block_expiry_seconds (integer, Minimum: 0, Maximum: 2147483647). The duration of time that login remains blocked after a failed login.max_blocked_clients (integer, Minimum: 0, Maximum: 2147483647). internal_authentication_backend_limiting.max_blocked_clients.max_tracked_clients (integer, Minimum: 0, Maximum: 2147483647). The maximum number of tracked IP addresses that have failed login.time_window_seconds (integer, Minimum: 0, Maximum: 2147483647). The window of time in which the value for allowed_tries is enforced.type (string, Enum: username, MaxLength: 1024). internal_authentication_backend_limiting.type.Appears on spec.userConfig.opensearch.auth_failure_listeners.
IP address rate limiting settings.
Optional
allowed_tries (integer, Minimum: 1, Maximum: 2147483647). The number of login attempts allowed before login is blocked.block_expiry_seconds (integer, Minimum: 1, Maximum: 36000). The duration of time that login remains blocked after a failed login.max_blocked_clients (integer, Minimum: 0, Maximum: 2147483647). The maximum number of blocked IP addresses.max_tracked_clients (integer, Minimum: 0, Maximum: 2147483647). The maximum number of tracked IP addresses that have failed login.time_window_seconds (integer, Minimum: 1, Maximum: 36000). The window of time in which the value for allowed_tries is enforced.type (string, Enum: ip, MaxLength: 1024). The type of rate limiting.Appears on spec.userConfig.
OpenSearch Dashboards settings.
Optional
enabled (boolean). Enable or disable OpenSearch Dashboards.max_old_space_size (integer, Minimum: 64, Maximum: 2048). Limits the maximum amount of memory (in MiB) the OpenSearch Dashboards process can use. This sets the max_old_space_size option of the nodejs running the OpenSearch Dashboards. Note: the memory reserved by OpenSearch Dashboards is not available for OpenSearch.opensearch_request_timeout (integer, Minimum: 5000, Maximum: 120000). Timeout in milliseconds for requests made by OpenSearch Dashboards towards OpenSearch.Appears on spec.userConfig.
Allow access to selected service ports from private networks.
Optional
opensearch (boolean). Allow clients to connect to opensearch with a DNS name that always resolves to the service's private IP addresses. Only available in certain network locations.opensearch_dashboards (boolean). Allow clients to connect to opensearch_dashboards with a DNS name that always resolves to the service's private IP addresses. Only available in certain network locations.prometheus (boolean). Allow clients to connect to prometheus with a DNS name that always resolves to the service's private IP addresses. Only available in certain network locations.Appears on spec.userConfig.
Allow access to selected service components through Privatelink.
Optional
opensearch (boolean). Enable opensearch.opensearch_dashboards (boolean). Enable opensearch_dashboards.prometheus (boolean). Enable prometheus.Appears on spec.userConfig.
Allow access to selected service ports from the public Internet.
Optional
opensearch (boolean). Allow clients to connect to opensearch from the public internet for service nodes that are in a project VPC or another type of private network.opensearch_dashboards (boolean). Allow clients to connect to opensearch_dashboards from the public internet for service nodes that are in a project VPC or another type of private network.prometheus (boolean). Allow clients to connect to prometheus from the public internet for service nodes that are in a project VPC or another type of private network.Appears on spec.userConfig.
OpenSearch SAML configuration.
Required
enabled (boolean). Enables or disables SAML-based authentication for OpenSearch. When enabled, users can authenticate using SAML with an Identity Provider.idp_entity_id (string, Pattern: ^[^\\r\\n]*$, MinLength: 1, MaxLength: 1024). The unique identifier for the Identity Provider (IdP) entity that is used for SAML authentication. This value is typically provided by the IdP.idp_metadata_url (string, MinLength: 1, MaxLength: 2048). The URL of the SAML metadata for the Identity Provider (IdP). This is used to configure SAML-based authentication with the IdP.sp_entity_id (string, Pattern: ^[^\\r\\n]*$, MinLength: 1, MaxLength: 1024). The unique identifier for the Service Provider (SP) entity that is used for SAML authentication. This value is typically provided by the SP.Optional
idp_pemtrustedcas_content (string, MaxLength: 16384). This parameter specifies the PEM-encoded root certificate authority (CA) content for the SAML identity provider (IdP) server verification. The root CA content is used to verify the SSL/TLS certificate presented by the server.roles_key (string, Pattern: ^[^\\r\\n]*$, MinLength: 1, MaxLength: 256). Optional. Specifies the attribute in the SAML response where role information is stored, if available. Role attributes are not required for SAML authentication, but can be included in SAML assertions by most Identity Providers (IdPs) to determine user access levels or permissions.subject_key (string, Pattern: ^[^\\r\\n]*$, MinLength: 1, MaxLength: 256). Optional. Specifies the attribute in the SAML response where the subject identifier is stored. If not configured, the NameID attribute is used by default.apiVersion: aiven.io/v1alpha1\nkind: PostgreSQL\nmetadata:\n name: my-postgresql\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n connInfoSecretTarget:\n name: postgresql-secret\n prefix: MY_SECRET_PREFIX_\n annotations:\n foo: bar\n labels:\n baz: egg\n\n project: aiven-project-name\n cloudName: google-europe-west1\n plan: startup-4\n\n maintenanceWindowDow: sunday\n maintenanceWindowTime: 11:00:00\n\n userConfig:\n pg_version: \"15\"\nInfo
To create this resource, a Secret containing Aiven token must be created first.
Apply the resource with:
kubectl apply -f example.yaml\nVerify the newly created PostgreSQL:
kubectl get postgresqls my-postgresql\nThe output is similar to the following:
Name Project Region Plan State \nmy-postgresql aiven-project-name google-europe-west1 startup-4 RUNNING \nTo view the details of the Secret, use the following command:
kubectl describe secret postgresql-secret\nYou can use the jq to quickly decode the Secret:
kubectl get secret postgresql-secret -o json | jq '.data | map_values(@base64d)'\nThe output is similar to the following:
{\n \"POSTGRESQL_HOST\": \"<secret>\",\n \"POSTGRESQL_PORT\": \"<secret>\",\n \"POSTGRESQL_DATABASE\": \"<secret>\",\n \"POSTGRESQL_USER\": \"<secret>\",\n \"POSTGRESQL_PASSWORD\": \"<secret>\",\n \"POSTGRESQL_SSLMODE\": \"<secret>\",\n \"POSTGRESQL_DATABASE_URI\": \"<secret>\",\n \"POSTGRESQL_CA_CERT\": \"<secret>\",\n}\nPostgreSQL is the Schema for the postgresql API.
Exposes secret keys
POSTGRESQL_HOST, POSTGRESQL_PORT, POSTGRESQL_DATABASE, POSTGRESQL_USER, POSTGRESQL_PASSWORD, POSTGRESQL_SSLMODE, POSTGRESQL_DATABASE_URI, POSTGRESQL_CA_CERT.
Required
apiVersion (string). Value aiven.io/v1alpha1.kind (string). Value PostgreSQL.metadata (object). Data that identifies the object, including a name string and optional namespace.spec (object). PostgreSQLSpec defines the desired state of postgres instance. See below for nested schema.Appears on PostgreSQL.
PostgreSQLSpec defines the desired state of postgres instance.
Required
plan (string, MaxLength: 128). Subscription plan.project (string, Immutable, Pattern: ^[a-zA-Z0-9_-]+$, MaxLength: 63). Identifies the project this resource belongs to.Optional
authSecretRef (object). Authentication reference to Aiven token in a secret. See below for nested schema.cloudName (string, MaxLength: 256). Cloud the service runs in.connInfoSecretTarget (object). Secret configuration. See below for nested schema.connInfoSecretTargetDisabled (boolean, Immutable). When true, the secret containing connection information will not be created, defaults to false. This field cannot be changed after resource creation.disk_space (string, Pattern: (?i)^[1-9][0-9]*(GiB|G)?$). The disk space of the service, possible values depend on the service type, the cloud provider and the project. Reducing will result in the service re-balancing. The removal of this field does not change the value.maintenanceWindowDow (string, Enum: monday, tuesday, wednesday, thursday, friday, saturday, sunday). Day of week when maintenance operations should be performed. One monday, tuesday, wednesday, etc.maintenanceWindowTime (string, MaxLength: 8). Time of day when maintenance operations should be performed. UTC time in HH:mm:ss format.projectVPCRef (object). ProjectVPCRef reference to ProjectVPC resource to use its ID as ProjectVPCID automatically. See below for nested schema.projectVpcId (string, MaxLength: 36). Identifier of the VPC the service should be in, if any.serviceIntegrations (array of objects, Immutable, MaxItems: 1). Service integrations to specify when creating a service. Not applied after initial service creation. See below for nested schema.tags (object, AdditionalProperties: string). Tags are key-value pairs that allow you to categorize services.technicalEmails (array of objects, MaxItems: 10). Defines the email addresses that will receive alerts about upcoming maintenance updates or warnings about service instability. See below for nested schema.terminationProtection (boolean). Prevent service from being deleted. It is recommended to have this enabled for all services.userConfig (object). PostgreSQL specific user configuration options. See below for nested schema.Appears on spec.
Authentication reference to Aiven token in a secret.
Required
key (string, MinLength: 1).name (string, MinLength: 1).Appears on spec.
Secret configuration.
Required
name (string, Immutable). Name of the secret resource to be created. By default, it is equal to the resource name.Optional
annotations (object, AdditionalProperties: string). Annotations added to the secret.labels (object, AdditionalProperties: string). Labels added to the secret.prefix (string). Prefix for the secret's keys. Added \"as is\" without any transformations. By default, is equal to the kind name in uppercase + underscore, e.g. KAFKA_, REDIS_, etc.Appears on spec.
ProjectVPCRef reference to ProjectVPC resource to use its ID as ProjectVPCID automatically.
Required
name (string, MinLength: 1).Optional
namespace (string, MinLength: 1).Appears on spec.
Service integrations to specify when creating a service. Not applied after initial service creation.
Required
integrationType (string, Enum: read_replica).sourceServiceName (string, MinLength: 1, MaxLength: 64).Appears on spec.
Defines the email addresses that will receive alerts about upcoming maintenance updates or warnings about service instability.
Required
email (string). Email address.Appears on spec.
PostgreSQL specific user configuration options.
Optional
additional_backup_regions (array of strings, MaxItems: 1). Additional Cloud Regions for Backup Replication.admin_password (string, Immutable, Pattern: ^[a-zA-Z0-9-_]+$, MinLength: 8, MaxLength: 256). Custom password for admin user. Defaults to random string. This must be set only when a new service is being created.admin_username (string, Immutable, Pattern: ^[_A-Za-z0-9][-._A-Za-z0-9]{0,63}$, MaxLength: 64). Custom username for admin user. This must be set only when a new service is being created.backup_hour (integer, Minimum: 0, Maximum: 23). The hour of day (in UTC) when backup for the service is started. New backup is only started if previous backup has already completed.backup_minute (integer, Minimum: 0, Maximum: 59). The minute of an hour when backup for the service is started. New backup is only started if previous backup has already completed.enable_ipv6 (boolean). Register AAAA DNS records for the service, and allow IPv6 packets to service ports.ip_filter (array of objects, MaxItems: 1024). Allow incoming connections from CIDR address block, e.g. 10.20.0.0/16. See below for nested schema.migration (object). Migrate data from existing server. See below for nested schema.pg (object). postgresql.conf configuration values. See below for nested schema.pg_qualstats (object). Deprecated. System-wide settings for the pg_qualstats extension. See below for nested schema.pg_read_replica (boolean). Should the service which is being forked be a read replica (deprecated, use read_replica service integration instead).pg_service_to_fork_from (string, Immutable, Pattern: ^[a-z][-a-z0-9]{0,63}$|^$, MaxLength: 64). Name of the PG Service from which to fork (deprecated, use service_to_fork_from). This has effect only when a new service is being created.pg_stat_monitor_enable (boolean). Enable the pg_stat_monitor extension. Enabling this extension will cause the cluster to be restarted.When this extension is enabled, pg_stat_statements results for utility commands are unreliable.pg_version (string, Enum: 12, 13, 14, 15, 16). PostgreSQL major version.pgaudit (object). Deprecated. System-wide settings for the pgaudit extension. See below for nested schema.pgbouncer (object). PGBouncer connection pooling settings. See below for nested schema.pglookout (object). System-wide settings for pglookout. See below for nested schema.private_access (object). Allow access to selected service ports from private networks. See below for nested schema.privatelink_access (object). Allow access to selected service components through Privatelink. See below for nested schema.project_to_fork_from (string, Immutable, Pattern: ^[a-z][-a-z0-9]{0,63}$|^$, MaxLength: 63). Name of another project to fork a service from. This has effect only when a new service is being created.public_access (object). Allow access to selected service ports from the public Internet. See below for nested schema.recovery_target_time (string, Immutable, MaxLength: 32). Recovery target time when forking a service. This has effect only when a new service is being created.service_log (boolean). Store logs for the service so that they are available in the HTTP API and console.service_to_fork_from (string, Immutable, Pattern: ^[a-z][-a-z0-9]{0,63}$|^$, MaxLength: 64). Name of another service to fork from. This has effect only when a new service is being created.shared_buffers_percentage (number, Minimum: 20, Maximum: 60). Percentage of total RAM that the database server uses for shared memory buffers. Valid range is 20-60 (float), which corresponds to 20% - 60%. This setting adjusts the shared_buffers configuration value.static_ips (boolean). Use static public IP addresses.synchronous_replication (string, Enum: quorum, off). Synchronous replication type. Note that the service plan also needs to support synchronous replication.timescaledb (object). System-wide settings for the timescaledb extension. See below for nested schema.variant (string, Enum: aiven, timescale). Variant of the PostgreSQL service, may affect the features that are exposed by default.work_mem (integer, Minimum: 1, Maximum: 1024). Sets the maximum amount of memory to be used by a query operation (such as a sort or hash table) before writing to temporary disk files, in MB. Default is 1MB + 0.075% of total RAM (up to 32MB).Appears on spec.userConfig.
CIDR address block, either as a string, or in a dict with an optional description field.
Required
network (string, MaxLength: 43). CIDR address block.Optional
description (string, MaxLength: 1024). Description for IP filter list entry.Appears on spec.userConfig.
Migrate data from existing server.
Required
host (string, MaxLength: 255). Hostname or IP address of the server where to migrate data from.port (integer, Minimum: 1, Maximum: 65535). Port number of the server where to migrate data from.Optional
dbname (string, MaxLength: 63). Database name for bootstrapping the initial connection.ignore_dbs (string, MaxLength: 2048). Comma-separated list of databases, which should be ignored during migration (supported by MySQL and PostgreSQL only at the moment).method (string, Enum: dump, replication). The migration method to be used (currently supported only by Redis, Dragonfly, MySQL and PostgreSQL service types).password (string, MaxLength: 256). Password for authentication with the server where to migrate data from.ssl (boolean). The server where to migrate data from is secured with SSL.username (string, MaxLength: 256). User name for authentication with the server where to migrate data from.Appears on spec.userConfig.
postgresql.conf configuration values.
Optional
autovacuum_analyze_scale_factor (number, Minimum: 0, Maximum: 1). Specifies a fraction of the table size to add to autovacuum_analyze_threshold when deciding whether to trigger an ANALYZE. The default is 0.2 (20% of table size).autovacuum_analyze_threshold (integer, Minimum: 0, Maximum: 2147483647). Specifies the minimum number of inserted, updated or deleted tuples needed to trigger an ANALYZE in any one table. The default is 50 tuples.autovacuum_freeze_max_age (integer, Minimum: 200000000, Maximum: 1500000000). Specifies the maximum age (in transactions) that a table's pg_class.relfrozenxid field can attain before a VACUUM operation is forced to prevent transaction ID wraparound within the table. Note that the system will launch autovacuum processes to prevent wraparound even when autovacuum is otherwise disabled. This parameter will cause the server to be restarted.autovacuum_max_workers (integer, Minimum: 1, Maximum: 20). Specifies the maximum number of autovacuum processes (other than the autovacuum launcher) that may be running at any one time. The default is three. This parameter can only be set at server start.autovacuum_naptime (integer, Minimum: 1, Maximum: 86400). Specifies the minimum delay between autovacuum runs on any given database. The delay is measured in seconds, and the default is one minute.autovacuum_vacuum_cost_delay (integer, Minimum: -1, Maximum: 100). Specifies the cost delay value that will be used in automatic VACUUM operations. If -1 is specified, the regular vacuum_cost_delay value will be used. The default value is 20 milliseconds.autovacuum_vacuum_cost_limit (integer, Minimum: -1, Maximum: 10000). Specifies the cost limit value that will be used in automatic VACUUM operations. If -1 is specified (which is the default), the regular vacuum_cost_limit value will be used.autovacuum_vacuum_scale_factor (number, Minimum: 0, Maximum: 1). Specifies a fraction of the table size to add to autovacuum_vacuum_threshold when deciding whether to trigger a VACUUM. The default is 0.2 (20% of table size).autovacuum_vacuum_threshold (integer, Minimum: 0, Maximum: 2147483647). Specifies the minimum number of updated or deleted tuples needed to trigger a VACUUM in any one table. The default is 50 tuples.bgwriter_delay (integer, Minimum: 10, Maximum: 10000). Specifies the delay between activity rounds for the background writer in milliseconds. Default is 200.bgwriter_flush_after (integer, Minimum: 0, Maximum: 2048). Whenever more than bgwriter_flush_after bytes have been written by the background writer, attempt to force the OS to issue these writes to the underlying storage. Specified in kilobytes, default is 512. Setting of 0 disables forced writeback.bgwriter_lru_maxpages (integer, Minimum: 0, Maximum: 1073741823). In each round, no more than this many buffers will be written by the background writer. Setting this to zero disables background writing. Default is 100.bgwriter_lru_multiplier (number, Minimum: 0, Maximum: 10). The average recent need for new buffers is multiplied by bgwriter_lru_multiplier to arrive at an estimate of the number that will be needed during the next round, (up to bgwriter_lru_maxpages). 1.0 represents a \u201cjust in time\u201d policy of writing exactly the number of buffers predicted to be needed. Larger values provide some cushion against spikes in demand, while smaller values intentionally leave writes to be done by server processes. The default is 2.0.deadlock_timeout (integer, Minimum: 500, Maximum: 1800000). This is the amount of time, in milliseconds, to wait on a lock before checking to see if there is a deadlock condition.default_toast_compression (string, Enum: lz4, pglz). Specifies the default TOAST compression method for values of compressible columns (the default is lz4).idle_in_transaction_session_timeout (integer, Minimum: 0, Maximum: 604800000). Time out sessions with open transactions after this number of milliseconds.jit (boolean). Controls system-wide use of Just-in-Time Compilation (JIT).log_autovacuum_min_duration (integer, Minimum: -1, Maximum: 2147483647). Causes each action executed by autovacuum to be logged if it ran for at least the specified number of milliseconds. Setting this to zero logs all autovacuum actions. Minus-one (the default) disables logging autovacuum actions.log_error_verbosity (string, Enum: TERSE, DEFAULT, VERBOSE). Controls the amount of detail written in the server log for each message that is logged.log_line_prefix (string, Enum: 'pid=%p,user=%u,db=%d,app=%a,client=%h ', '%t [%p]: [%l-1] user=%u,db=%d,app=%a,client=%h ', '%m [%p] %q[user=%u,db=%d,app=%a] ', 'pid=%p,user=%u,db=%d,app=%a,client=%h,txid=%x,qid=%Q '). Choose from one of the available log formats.log_min_duration_statement (integer, Minimum: -1, Maximum: 86400000). Log statements that take more than this number of milliseconds to run, -1 disables.log_temp_files (integer, Minimum: -1, Maximum: 2147483647). Log statements for each temporary file created larger than this number of kilobytes, -1 disables.max_files_per_process (integer, Minimum: 1000, Maximum: 4096). PostgreSQL maximum number of files that can be open per process.max_locks_per_transaction (integer, Minimum: 64, Maximum: 6400). PostgreSQL maximum locks per transaction.max_logical_replication_workers (integer, Minimum: 4, Maximum: 64). PostgreSQL maximum logical replication workers (taken from the pool of max_parallel_workers).max_parallel_workers (integer, Minimum: 0, Maximum: 96). Sets the maximum number of workers that the system can support for parallel queries.max_parallel_workers_per_gather (integer, Minimum: 0, Maximum: 96). Sets the maximum number of workers that can be started by a single Gather or Gather Merge node.max_pred_locks_per_transaction (integer, Minimum: 64, Maximum: 5120). PostgreSQL maximum predicate locks per transaction.max_prepared_transactions (integer, Minimum: 0, Maximum: 10000). PostgreSQL maximum prepared transactions.max_replication_slots (integer, Minimum: 8, Maximum: 64). PostgreSQL maximum replication slots.max_slot_wal_keep_size (integer, Minimum: -1, Maximum: 2147483647). PostgreSQL maximum WAL size (MB) reserved for replication slots. Default is -1 (unlimited). wal_keep_size minimum WAL size setting takes precedence over this.max_stack_depth (integer, Minimum: 2097152, Maximum: 6291456). Maximum depth of the stack in bytes.max_standby_archive_delay (integer, Minimum: 1, Maximum: 43200000). Max standby archive delay in milliseconds.max_standby_streaming_delay (integer, Minimum: 1, Maximum: 43200000). Max standby streaming delay in milliseconds.max_wal_senders (integer, Minimum: 20, Maximum: 64). PostgreSQL maximum WAL senders.max_worker_processes (integer, Minimum: 8, Maximum: 96). Sets the maximum number of background processes that the system can support.pg_partman_bgw.interval (integer, Minimum: 3600, Maximum: 604800). Sets the time interval to run pg_partman's scheduled tasks.pg_partman_bgw.role (string, Pattern: ^[_A-Za-z0-9][-._A-Za-z0-9]{0,63}$, MaxLength: 64). Controls which role to use for pg_partman's scheduled background tasks.pg_stat_monitor.pgsm_enable_query_plan (boolean). Enables or disables query plan monitoring.pg_stat_monitor.pgsm_max_buckets (integer, Minimum: 1, Maximum: 10). Sets the maximum number of buckets.pg_stat_statements.track (string, Enum: all, top, none). Controls which statements are counted. Specify top to track top-level statements (those issued directly by clients), all to also track nested statements (such as statements invoked within functions), or none to disable statement statistics collection. The default value is top.temp_file_limit (integer, Minimum: -1, Maximum: 2147483647). PostgreSQL temporary file limit in KiB, -1 for unlimited.timezone (string, Pattern: ^[\\w/]*$, MaxLength: 64). PostgreSQL service timezone.track_activity_query_size (integer, Minimum: 1024, Maximum: 10240). Specifies the number of bytes reserved to track the currently executing command for each active session.track_commit_timestamp (string, Enum: off, on). Record commit time of transactions.track_functions (string, Enum: all, pl, none). Enables tracking of function call counts and time used.track_io_timing (string, Enum: off, on). Enables timing of database I/O calls. This parameter is off by default, because it will repeatedly query the operating system for the current time, which may cause significant overhead on some platforms.wal_sender_timeout (integer). Terminate replication connections that are inactive for longer than this amount of time, in milliseconds. Setting this value to zero disables the timeout.wal_writer_delay (integer, Minimum: 10, Maximum: 200). WAL flush interval in milliseconds. Note that setting this value to lower than the default 200ms may negatively impact performance.Appears on spec.userConfig.
Deprecated. System-wide settings for the pg_qualstats extension.
Optional
enabled (boolean). Deprecated. Enable / Disable pg_qualstats.min_err_estimate_num (integer, Minimum: 0). Deprecated. Error estimation num threshold to save quals.min_err_estimate_ratio (integer, Minimum: 0). Deprecated. Error estimation ratio threshold to save quals.track_constants (boolean). Deprecated. Enable / Disable pg_qualstats constants tracking.track_pg_catalog (boolean). Deprecated. Track quals on system catalogs too.Appears on spec.userConfig.
Deprecated. System-wide settings for the pgaudit extension.
Optional
feature_enabled (boolean). Deprecated. Enable pgaudit extension. When enabled, pgaudit extension will be automatically installed.Otherwise, extension will be uninstalled but auditing configurations will be preserved.log (array of strings). Deprecated. Specifies which classes of statements will be logged by session audit logging.log_catalog (boolean). Deprecated. Specifies that session logging should be enabled in the casewhere all relations in a statement are in pg_catalog.log_client (boolean). Deprecated. Specifies whether log messages will be visible to a client process such as psql.log_level (string). Deprecated. Specifies the log level that will be used for log entries.log_max_string_length (integer, Minimum: -1, Maximum: 102400). Deprecated. Crop parameters representation and whole statements if they exceed this threshold. A (default) value of -1 disable the truncation.log_nested_statements (boolean). Deprecated. This GUC allows to turn off logging nested statements, that is, statements that are executed as part of another ExecutorRun.log_parameter (boolean). Deprecated. Specifies that audit logging should include the parameters that were passed with the statement.log_parameter_max_size (integer). Deprecated. Specifies that parameter values longer than this setting (in bytes) should not be logged, but replaced with . log_relation (boolean). Deprecated. Specifies whether session audit logging should create a separate log entry for each relation (TABLE, VIEW, etc.) referenced in a SELECT or DML statement.log_rows (boolean). Deprecated. Specifies that audit logging should include the rows retrieved or affected by a statement. When enabled the rows field will be included after the parameter field.log_statement (boolean). Deprecated. Specifies whether logging will include the statement text and parameters (if enabled).log_statement_once (boolean). Deprecated. Specifies whether logging will include the statement text and parameters with the first log entry for a statement/substatement combination or with every entry.role (string, Pattern: ^[_A-Za-z0-9][-._A-Za-z0-9]{0,63}$, MaxLength: 64). Deprecated. Specifies the master role to use for object audit logging.Appears on spec.userConfig.
PGBouncer connection pooling settings.
Optional
autodb_idle_timeout (integer, Minimum: 0, Maximum: 86400). If the automatically created database pools have been unused this many seconds, they are freed. If 0 then timeout is disabled. [seconds].autodb_max_db_connections (integer, Minimum: 0, Maximum: 2147483647). Do not allow more than this many server connections per database (regardless of user). Setting it to 0 means unlimited.autodb_pool_mode (string, Enum: session, transaction, statement). PGBouncer pool mode.autodb_pool_size (integer, Minimum: 0, Maximum: 10000). If non-zero then create automatically a pool of that size per user when a pool doesn't exist.ignore_startup_parameters (array of strings, MaxItems: 32). List of parameters to ignore when given in startup packet.min_pool_size (integer, Minimum: 0, Maximum: 10000). Add more server connections to pool if below this number. Improves behavior when usual load comes suddenly back after period of total inactivity. The value is effectively capped at the pool size.server_idle_timeout (integer, Minimum: 0, Maximum: 86400). If a server connection has been idle more than this many seconds it will be dropped. If 0 then timeout is disabled. [seconds].server_lifetime (integer, Minimum: 60, Maximum: 86400). The pooler will close an unused server connection that has been connected longer than this. [seconds].server_reset_query_always (boolean). Run server_reset_query (DISCARD ALL) in all pooling modes.Appears on spec.userConfig.
System-wide settings for pglookout.
Required
max_failover_replication_time_lag (integer, Minimum: 10). Number of seconds of master unavailability before triggering database failover to standby.Appears on spec.userConfig.
Allow access to selected service ports from private networks.
Optional
pg (boolean). Allow clients to connect to pg with a DNS name that always resolves to the service's private IP addresses. Only available in certain network locations.pgbouncer (boolean). Allow clients to connect to pgbouncer with a DNS name that always resolves to the service's private IP addresses. Only available in certain network locations.prometheus (boolean). Allow clients to connect to prometheus with a DNS name that always resolves to the service's private IP addresses. Only available in certain network locations.Appears on spec.userConfig.
Allow access to selected service components through Privatelink.
Optional
pg (boolean). Enable pg.pgbouncer (boolean). Enable pgbouncer.prometheus (boolean). Enable prometheus.Appears on spec.userConfig.
Allow access to selected service ports from the public Internet.
Optional
pg (boolean). Allow clients to connect to pg from the public internet for service nodes that are in a project VPC or another type of private network.pgbouncer (boolean). Allow clients to connect to pgbouncer from the public internet for service nodes that are in a project VPC or another type of private network.prometheus (boolean). Allow clients to connect to prometheus from the public internet for service nodes that are in a project VPC or another type of private network.Appears on spec.userConfig.
System-wide settings for the timescaledb extension.
Required
max_background_workers (integer, Minimum: 1, Maximum: 4096). The number of background workers for timescaledb operations. You should configure this setting to the sum of your number of databases and the total number of concurrent background workers you want running at any given point in time.apiVersion: aiven.io/v1alpha1\nkind: Project\nmetadata:\n name: my-project\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n connInfoSecretTarget:\n name: project-secret\n prefix: MY_SECRET_PREFIX_\n annotations:\n foo: bar\n labels:\n baz: egg\n\n tags:\n env: prod\n\n billingAddress: NYC\n cloud: aws-eu-west-1\nInfo
To create this resource, a Secret containing Aiven token must be created first.
Apply the resource with:
kubectl apply -f example.yaml\nVerify the newly created Project:
kubectl get projects my-project\nThe output is similar to the following:
Name \nmy-project \nTo view the details of the Secret, use the following command:
kubectl describe secret project-secret\nYou can use the jq to quickly decode the Secret:
kubectl get secret project-secret -o json | jq '.data | map_values(@base64d)'\nThe output is similar to the following:
{\n \"PROJECT_CA_CERT\": \"<secret>\",\n}\nProject is the Schema for the projects API.
Exposes secret keys
PROJECT_CA_CERT.
Required
apiVersion (string). Value aiven.io/v1alpha1.kind (string). Value Project.metadata (object). Data that identifies the object, including a name string and optional namespace.spec (object). ProjectSpec defines the desired state of Project. See below for nested schema.Appears on Project.
ProjectSpec defines the desired state of Project.
Optional
accountId (string, MaxLength: 32). Account ID.authSecretRef (object). Authentication reference to Aiven token in a secret. See below for nested schema.billingAddress (string, MaxLength: 1000). Billing name and address of the project.billingCurrency (string, Enum: AUD, CAD, CHF, DKK, EUR, GBP, NOK, SEK, USD). Billing currency.billingEmails (array of strings, MaxItems: 10). Billing contact emails of the project.billingExtraText (string, MaxLength: 1000). Extra text to be included in all project invoices, e.g. purchase order or cost center number.billingGroupId (string, Immutable, MinLength: 36, MaxLength: 36). BillingGroup ID.cardId (string, MaxLength: 64). Credit card ID; The ID may be either last 4 digits of the card or the actual ID.cloud (string, MaxLength: 256). Target cloud, example: aws-eu-central-1.connInfoSecretTarget (object). Secret configuration. See below for nested schema.connInfoSecretTargetDisabled (boolean, Immutable). When true, the secret containing connection information will not be created, defaults to false. This field cannot be changed after resource creation.copyFromProject (string, Immutable, MaxLength: 63). Project name from which to copy settings to the new project.countryCode (string, MinLength: 2, MaxLength: 2). Billing country code of the project.tags (object, AdditionalProperties: string). Tags are key-value pairs that allow you to categorize projects.technicalEmails (array of strings, MaxItems: 10). Technical contact emails of the project.Appears on spec.
Authentication reference to Aiven token in a secret.
Required
key (string, MinLength: 1).name (string, MinLength: 1).Appears on spec.
Secret configuration.
Required
name (string, Immutable). Name of the secret resource to be created. By default, it is equal to the resource name.Optional
annotations (object, AdditionalProperties: string). Annotations added to the secret.labels (object, AdditionalProperties: string). Labels added to the secret.prefix (string). Prefix for the secret's keys. Added \"as is\" without any transformations. By default, is equal to the kind name in uppercase + underscore, e.g. KAFKA_, REDIS_, etc.apiVersion: aiven.io/v1alpha1\nkind: ProjectVPC\nmetadata:\n name: my-project-vpc\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n project: aiven-project-name\n cloudName: google-europe-west1\n networkCidr: 10.0.0.0/24\nInfo
To create this resource, a Secret containing Aiven token must be created first.
Apply the resource with:
kubectl apply -f example.yaml\nVerify the newly created ProjectVPC:
kubectl get projectvpcs my-project-vpc\nThe output is similar to the following:
Name Project Cloud Network CIDR State \nmy-project-vpc aiven-project-name google-europe-west1 10.0.0.0/24 RUNNING \nProjectVPC is the Schema for the projectvpcs API.
Required
apiVersion (string). Value aiven.io/v1alpha1.kind (string). Value ProjectVPC.metadata (object). Data that identifies the object, including a name string and optional namespace.spec (object). ProjectVPCSpec defines the desired state of ProjectVPC. See below for nested schema.Appears on ProjectVPC.
ProjectVPCSpec defines the desired state of ProjectVPC.
Required
cloudName (string, Immutable, MaxLength: 256). Cloud the VPC is in.networkCidr (string, Immutable, MaxLength: 36). Network address range used by the VPC like 192.168.0.0/24.project (string, Immutable, Pattern: ^[a-zA-Z0-9_-]+$, MaxLength: 63). Identifies the project this resource belongs to.Optional
authSecretRef (object). Authentication reference to Aiven token in a secret. See below for nested schema.Appears on spec.
Authentication reference to Aiven token in a secret.
Required
key (string, MinLength: 1).name (string, MinLength: 1).apiVersion: aiven.io/v1alpha1\nkind: Redis\nmetadata:\n name: k8s-redis\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n connInfoSecretTarget:\n name: redis-token\n prefix: MY_SECRET_PREFIX_\n annotations:\n foo: bar\n labels:\n baz: egg\n\n project: my-aiven-project\n cloudName: google-europe-west1\n plan: startup-4\n\n maintenanceWindowDow: friday\n maintenanceWindowTime: 23:00:00\n\n userConfig:\n redis_maxmemory_policy: allkeys-random\nInfo
To create this resource, a Secret containing Aiven token must be created first.
Apply the resource with:
kubectl apply -f example.yaml\nVerify the newly created Redis:
kubectl get redis k8s-redis\nThe output is similar to the following:
Name Project Region Plan State \nk8s-redis my-aiven-project google-europe-west1 startup-4 RUNNING \nTo view the details of the Secret, use the following command:
kubectl describe secret redis-token\nYou can use the jq to quickly decode the Secret:
kubectl get secret redis-token -o json | jq '.data | map_values(@base64d)'\nThe output is similar to the following:
{\n \"REDIS_HOST\": \"<secret>\",\n \"REDIS_PORT\": \"<secret>\",\n \"REDIS_USER\": \"<secret>\",\n \"REDIS_PASSWORD\": \"<secret>\",\n}\nRedis is the Schema for the redis API.
Exposes secret keys
REDIS_HOST, REDIS_PORT, REDIS_USER, REDIS_PASSWORD.
Required
apiVersion (string). Value aiven.io/v1alpha1.kind (string). Value Redis.metadata (object). Data that identifies the object, including a name string and optional namespace.spec (object). RedisSpec defines the desired state of Redis. See below for nested schema.Appears on Redis.
RedisSpec defines the desired state of Redis.
Required
plan (string, MaxLength: 128). Subscription plan.project (string, Immutable, Pattern: ^[a-zA-Z0-9_-]+$, MaxLength: 63). Identifies the project this resource belongs to.Optional
authSecretRef (object). Authentication reference to Aiven token in a secret. See below for nested schema.cloudName (string, MaxLength: 256). Cloud the service runs in.connInfoSecretTarget (object). Secret configuration. See below for nested schema.connInfoSecretTargetDisabled (boolean, Immutable). When true, the secret containing connection information will not be created, defaults to false. This field cannot be changed after resource creation.disk_space (string, Pattern: (?i)^[1-9][0-9]*(GiB|G)?$). The disk space of the service, possible values depend on the service type, the cloud provider and the project. Reducing will result in the service re-balancing. The removal of this field does not change the value.maintenanceWindowDow (string, Enum: monday, tuesday, wednesday, thursday, friday, saturday, sunday). Day of week when maintenance operations should be performed. One monday, tuesday, wednesday, etc.maintenanceWindowTime (string, MaxLength: 8). Time of day when maintenance operations should be performed. UTC time in HH:mm:ss format.projectVPCRef (object). ProjectVPCRef reference to ProjectVPC resource to use its ID as ProjectVPCID automatically. See below for nested schema.projectVpcId (string, MaxLength: 36). Identifier of the VPC the service should be in, if any.serviceIntegrations (array of objects, Immutable, MaxItems: 1). Service integrations to specify when creating a service. Not applied after initial service creation. See below for nested schema.tags (object, AdditionalProperties: string). Tags are key-value pairs that allow you to categorize services.technicalEmails (array of objects, MaxItems: 10). Defines the email addresses that will receive alerts about upcoming maintenance updates or warnings about service instability. See below for nested schema.terminationProtection (boolean). Prevent service from being deleted. It is recommended to have this enabled for all services.userConfig (object). Redis specific user configuration options. See below for nested schema.Appears on spec.
Authentication reference to Aiven token in a secret.
Required
key (string, MinLength: 1).name (string, MinLength: 1).Appears on spec.
Secret configuration.
Required
name (string, Immutable). Name of the secret resource to be created. By default, it is equal to the resource name.Optional
annotations (object, AdditionalProperties: string). Annotations added to the secret.labels (object, AdditionalProperties: string). Labels added to the secret.prefix (string). Prefix for the secret's keys. Added \"as is\" without any transformations. By default, is equal to the kind name in uppercase + underscore, e.g. KAFKA_, REDIS_, etc.Appears on spec.
ProjectVPCRef reference to ProjectVPC resource to use its ID as ProjectVPCID automatically.
Required
name (string, MinLength: 1).Optional
namespace (string, MinLength: 1).Appears on spec.
Service integrations to specify when creating a service. Not applied after initial service creation.
Required
integrationType (string, Enum: read_replica).sourceServiceName (string, MinLength: 1, MaxLength: 64).Appears on spec.
Defines the email addresses that will receive alerts about upcoming maintenance updates or warnings about service instability.
Required
email (string). Email address.Appears on spec.
Redis specific user configuration options.
Optional
additional_backup_regions (array of strings, MaxItems: 1). Additional Cloud Regions for Backup Replication.ip_filter (array of objects, MaxItems: 1024). Allow incoming connections from CIDR address block, e.g. 10.20.0.0/16. See below for nested schema.migration (object). Migrate data from existing server. See below for nested schema.private_access (object). Allow access to selected service ports from private networks. See below for nested schema.privatelink_access (object). Allow access to selected service components through Privatelink. See below for nested schema.project_to_fork_from (string, Immutable, Pattern: ^[a-z][-a-z0-9]{0,63}$|^$, MaxLength: 63). Name of another project to fork a service from. This has effect only when a new service is being created.public_access (object). Allow access to selected service ports from the public Internet. See below for nested schema.recovery_basebackup_name (string, Pattern: ^[a-zA-Z0-9-_:.]+$, MaxLength: 128). Name of the basebackup to restore in forked service.redis_acl_channels_default (string, Enum: allchannels, resetchannels). Determines default pub/sub channels' ACL for new users if ACL is not supplied. When this option is not defined, all_channels is assumed to keep backward compatibility. This option doesn't affect Redis configuration acl-pubsub-default.redis_io_threads (integer, Minimum: 1, Maximum: 32). Set Redis IO thread count. Changing this will cause a restart of the Redis service.redis_lfu_decay_time (integer, Minimum: 1, Maximum: 120). LFU maxmemory-policy counter decay time in minutes.redis_lfu_log_factor (integer, Minimum: 0, Maximum: 100). Counter logarithm factor for volatile-lfu and allkeys-lfu maxmemory-policies.redis_maxmemory_policy (string, Enum: noeviction, allkeys-lru, volatile-lru, allkeys-random, volatile-random, volatile-ttl, volatile-lfu, allkeys-lfu). Redis maxmemory-policy.redis_notify_keyspace_events (string, Pattern: ^[KEg\\$lshzxentdmA]*$, MaxLength: 32). Set notify-keyspace-events option.redis_number_of_databases (integer, Minimum: 1, Maximum: 128). Set number of Redis databases. Changing this will cause a restart of the Redis service.redis_persistence (string, Enum: off, rdb). When persistence is rdb, Redis does RDB dumps each 10 minutes if any key is changed. Also RDB dumps are done according to the backup schedule for backup purposes. When persistence is off, no RDB dumps or backups are done, so data can be lost at any moment if the service is restarted for any reason, or if the service is powered off. Also, the service can't be forked.redis_pubsub_client_output_buffer_limit (integer, Minimum: 32, Maximum: 512). Set output buffer limit for pub / sub clients in MB. The value is the hard limit, the soft limit is 1/4 of the hard limit. When setting the limit, be mindful of the available memory in the selected service plan.redis_ssl (boolean). Require SSL to access Redis.redis_timeout (integer, Minimum: 0, Maximum: 31536000). Redis idle connection timeout in seconds.redis_version (string, Enum: 7.0). Redis major version.service_log (boolean). Store logs for the service so that they are available in the HTTP API and console.service_to_fork_from (string, Immutable, Pattern: ^[a-z][-a-z0-9]{0,63}$|^$, MaxLength: 64). Name of another service to fork from. This has effect only when a new service is being created.static_ips (boolean). Use static public IP addresses.Appears on spec.userConfig.
CIDR address block, either as a string, or in a dict with an optional description field.
Required
network (string, MaxLength: 43). CIDR address block.Optional
description (string, MaxLength: 1024). Description for IP filter list entry.Appears on spec.userConfig.
Migrate data from existing server.
Required
host (string, MaxLength: 255). Hostname or IP address of the server where to migrate data from.port (integer, Minimum: 1, Maximum: 65535). Port number of the server where to migrate data from.Optional
dbname (string, MaxLength: 63). Database name for bootstrapping the initial connection.ignore_dbs (string, MaxLength: 2048). Comma-separated list of databases, which should be ignored during migration (supported by MySQL and PostgreSQL only at the moment).method (string, Enum: dump, replication). The migration method to be used (currently supported only by Redis, Dragonfly, MySQL and PostgreSQL service types).password (string, MaxLength: 256). Password for authentication with the server where to migrate data from.ssl (boolean). The server where to migrate data from is secured with SSL.username (string, MaxLength: 256). User name for authentication with the server where to migrate data from.Appears on spec.userConfig.
Allow access to selected service ports from private networks.
Optional
prometheus (boolean). Allow clients to connect to prometheus with a DNS name that always resolves to the service's private IP addresses. Only available in certain network locations.redis (boolean). Allow clients to connect to redis with a DNS name that always resolves to the service's private IP addresses. Only available in certain network locations.Appears on spec.userConfig.
Allow access to selected service components through Privatelink.
Optional
prometheus (boolean). Enable prometheus.redis (boolean). Enable redis.Appears on spec.userConfig.
Allow access to selected service ports from the public Internet.
Optional
prometheus (boolean). Allow clients to connect to prometheus from the public internet for service nodes that are in a project VPC or another type of private network.redis (boolean). Allow clients to connect to redis from the public internet for service nodes that are in a project VPC or another type of private network.apiVersion: aiven.io/v1alpha1\nkind: ServiceIntegration\nmetadata:\n name: my-service-integration\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n project: aiven-project-name\n integrationType: clickhouse_postgresql\n sourceServiceName: my-pg\n destinationServiceName: my-clickhouse\n\n clickhousePostgresql:\n databases:\n - database: defaultdb\n schema: public\n\n---\n\napiVersion: aiven.io/v1alpha1\nkind: Clickhouse\nmetadata:\n name: my-clickhouse\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n project: aiven-project-name\n cloudName: google-europe-west1\n plan: startup-16\n maintenanceWindowDow: friday\n maintenanceWindowTime: 23:00:00\n\n---\n\napiVersion: aiven.io/v1alpha1\nkind: PostgreSQL\nmetadata:\n name: my-pg\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n project: aiven-project-name\n cloudName: google-europe-west1\n plan: startup-4\n maintenanceWindowDow: friday\n maintenanceWindowTime: 23:00:00\napiVersion: aiven.io/v1alpha1\nkind: ServiceIntegration\nmetadata:\n name: my-service-integration\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n project: aiven-project-name\n integrationType: datadog\n sourceServiceName: my-pg\n destinationEndpointId: destination-endpoint-id\n\n datadog:\n datadog_dbm_enabled: True\n datadog_tags:\n - tag: env\n comment: test\n\n---\n\napiVersion: aiven.io/v1alpha1\nkind: PostgreSQL\nmetadata:\n name: my-pg\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n project: aiven-project-name\n cloudName: google-europe-west1\n plan: startup-4\napiVersion: aiven.io/v1alpha1\nkind: ServiceIntegration\nmetadata:\n name: my-service-integration\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n project: aiven-project-name\n integrationType: kafka_connect\n sourceServiceName: my-kafka\n destinationServiceName: my-kafka-connect\n\n kafkaConnect:\n kafka_connect:\n group_id: connect\n status_storage_topic: __connect_status\n offset_storage_topic: __connect_offsets\n\n---\n\napiVersion: aiven.io/v1alpha1\nkind: Kafka\nmetadata:\n name: my-kafka\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n project: aiven-project-name\n cloudName: google-europe-west1\n plan: business-4\n\n---\n\napiVersion: aiven.io/v1alpha1\nkind: KafkaConnect\nmetadata:\n name: my-kafka-connect\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n project: aiven-project-name\n cloudName: google-europe-west1\n plan: business-4\n\n userConfig:\n kafka_connect:\n consumer_isolation_level: read_committed\n public_access:\n kafka_connect: true\napiVersion: aiven.io/v1alpha1\nkind: ServiceIntegration\nmetadata:\n name: my-service-integration\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n project: aiven-project-name\n integrationType: kafka_logs\n sourceServiceName: my-kafka\n destinationServiceName: my-kafka\n\n kafkaLogs:\n kafka_topic: my-kafka-topic\n\n---\n\napiVersion: aiven.io/v1alpha1\nkind: Kafka\nmetadata:\n name: my-kafka\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n project: aiven-project-name\n cloudName: google-europe-west1\n plan: business-4\n\n---\n\napiVersion: aiven.io/v1alpha1\nkind: KafkaTopic\nmetadata:\n name: my-kafka-topic\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n project: aiven-project-name\n serviceName: my-kafka\n replication: 2\n partitions: 1\nInfo
To create this resource, a Secret containing Aiven token must be created first.
Apply the resource with:
kubectl apply -f example.yaml\nVerify the newly created ServiceIntegration:
kubectl get serviceintegrations my-service-integration\nThe output is similar to the following:
Name Project Type Source Service Name Destination Service Name \nmy-service-integration aiven-project-name clickhouse_postgresql my-pg my-clickhouse \nServiceIntegration is the Schema for the serviceintegrations API.
Required
apiVersion (string). Value aiven.io/v1alpha1.kind (string). Value ServiceIntegration.metadata (object). Data that identifies the object, including a name string and optional namespace.spec (object). ServiceIntegrationSpec defines the desired state of ServiceIntegration. See below for nested schema.Appears on ServiceIntegration.
ServiceIntegrationSpec defines the desired state of ServiceIntegration.
Required
integrationType (string, Enum: alertmanager, autoscaler, caching, cassandra_cross_service_cluster, clickhouse_kafka, clickhouse_postgresql, dashboard, datadog, datasource, external_aws_cloudwatch_logs, external_aws_cloudwatch_metrics, external_elasticsearch_logs, external_google_cloud_logging, external_opensearch_logs, flink, flink_external_kafka, flink_external_postgresql, internal_connectivity, jolokia, kafka_connect, kafka_logs, kafka_mirrormaker, logs, m3aggregator, m3coordinator, metrics, opensearch_cross_cluster_replication, opensearch_cross_cluster_search, prometheus, read_replica, rsyslog, schema_registry_proxy, stresstester, thanosquery, thanosstore, vmalert, Immutable). Type of the service integration accepted by Aiven API. Some values may not be supported by the operator.project (string, Immutable, Pattern: ^[a-zA-Z0-9_-]+$, MaxLength: 63). Identifies the project this resource belongs to.Optional
authSecretRef (object). Authentication reference to Aiven token in a secret. See below for nested schema.clickhouseKafka (object). Clickhouse Kafka configuration values. See below for nested schema.clickhousePostgresql (object). Clickhouse PostgreSQL configuration values. See below for nested schema.datadog (object). Datadog specific user configuration options. See below for nested schema.destinationEndpointId (string, Immutable, MaxLength: 36). Destination endpoint for the integration (if any).destinationProjectName (string, Immutable, MaxLength: 63). Destination project for the integration (if any).destinationServiceName (string, Immutable, MaxLength: 64). Destination service for the integration (if any).externalAWSCloudwatchMetrics (object). External AWS CloudWatch Metrics integration Logs configuration values. See below for nested schema.kafkaConnect (object). Kafka Connect service configuration values. See below for nested schema.kafkaLogs (object). Kafka logs configuration values. See below for nested schema.kafkaMirrormaker (object). Kafka MirrorMaker configuration values. See below for nested schema.logs (object). Logs configuration values. See below for nested schema.metrics (object). Metrics configuration values. See below for nested schema.sourceEndpointID (string, Immutable, MaxLength: 36). Source endpoint for the integration (if any).sourceProjectName (string, Immutable, MaxLength: 63). Source project for the integration (if any).sourceServiceName (string, Immutable, MaxLength: 64). Source service for the integration (if any).Appears on spec.
Authentication reference to Aiven token in a secret.
Required
key (string, MinLength: 1).name (string, MinLength: 1).Appears on spec.
Clickhouse Kafka configuration values.
Required
tables (array of objects, MaxItems: 100). Tables to create. See below for nested schema.Appears on spec.clickhouseKafka.
Table to create.
Required
columns (array of objects, MaxItems: 100). Table columns. See below for nested schema.data_format (string, Enum: Avro, CSV, JSONAsString, JSONCompactEachRow, JSONCompactStringsEachRow, JSONEachRow, JSONStringsEachRow, MsgPack, TSKV, TSV, TabSeparated, RawBLOB, AvroConfluent, Parquet). Message data format.group_name (string, MinLength: 1, MaxLength: 249). Kafka consumers group.name (string, MinLength: 1, MaxLength: 40). Name of the table.topics (array of objects, MaxItems: 100). Kafka topics. See below for nested schema.Optional
auto_offset_reset (string, Enum: smallest, earliest, beginning, largest, latest, end). Action to take when there is no initial offset in offset store or the desired offset is out of range.date_time_input_format (string, Enum: basic, best_effort, best_effort_us). Method to read DateTime from text input formats.handle_error_mode (string, Enum: default, stream). How to handle errors for Kafka engine.max_block_size (integer, Minimum: 0, Maximum: 1000000000). Number of row collected by poll(s) for flushing data from Kafka.max_rows_per_message (integer, Minimum: 1, Maximum: 1000000000). The maximum number of rows produced in one kafka message for row-based formats.num_consumers (integer, Minimum: 1, Maximum: 10). The number of consumers per table per replica.poll_max_batch_size (integer, Minimum: 0, Maximum: 1000000000). Maximum amount of messages to be polled in a single Kafka poll.skip_broken_messages (integer, Minimum: 0, Maximum: 1000000000). Skip at least this number of broken messages from Kafka topic per block.Appears on spec.clickhouseKafka.tables.
Table column.
Required
name (string, MinLength: 1, MaxLength: 40). Column name.type (string, MinLength: 1, MaxLength: 1000). Column type.Appears on spec.clickhouseKafka.tables.
Kafka topic.
Required
name (string, MinLength: 1, MaxLength: 249). Name of the topic.Appears on spec.
Clickhouse PostgreSQL configuration values.
Required
databases (array of objects, MaxItems: 10). Databases to expose. See below for nested schema.Appears on spec.clickhousePostgresql.
Database to expose.
Optional
database (string, MinLength: 1, MaxLength: 63). PostgreSQL database to expose.schema (string, MinLength: 1, MaxLength: 63). PostgreSQL schema to expose.Appears on spec.
Datadog specific user configuration options.
Optional
datadog_dbm_enabled (boolean). Enable Datadog Database Monitoring.datadog_pgbouncer_enabled (boolean). Enable Datadog PgBouncer Metric Tracking.datadog_tags (array of objects, MaxItems: 32). Custom tags provided by user. See below for nested schema.exclude_consumer_groups (array of strings, MaxItems: 1024). List of custom metrics.exclude_topics (array of strings, MaxItems: 1024). List of topics to exclude.include_consumer_groups (array of strings, MaxItems: 1024). List of custom metrics.include_topics (array of strings, MaxItems: 1024). List of topics to include.kafka_custom_metrics (array of strings, MaxItems: 1024). List of custom metrics.max_jmx_metrics (integer, Minimum: 10, Maximum: 100000). Maximum number of JMX metrics to send.mirrormaker_custom_metrics (array of strings, MaxItems: 1024). List of custom metrics.opensearch (object). Datadog Opensearch Options. See below for nested schema.redis (object). Datadog Redis Options. See below for nested schema.Appears on spec.datadog.
Datadog tag defined by user.
Required
tag (string, MinLength: 1, MaxLength: 200). Tag format and usage are described here: https://docs.datadoghq.com/getting_started/tagging. Tags with prefix aiven- are reserved for Aiven.Optional
comment (string, MaxLength: 1024). Optional tag explanation.Appears on spec.datadog.
Datadog Opensearch Options.
Optional
cluster_stats_enabled (boolean). Enable Datadog Opensearch Cluster Monitoring.index_stats_enabled (boolean). Enable Datadog Opensearch Index Monitoring.pending_task_stats_enabled (boolean). Enable Datadog Opensearch Pending Task Monitoring.pshard_stats_enabled (boolean). Enable Datadog Opensearch Primary Shard Monitoring.Appears on spec.datadog.
Datadog Redis Options.
Required
command_stats_enabled (boolean). Enable command_stats option in the agent's configuration.Appears on spec.
External AWS CloudWatch Metrics integration Logs configuration values.
Optional
dropped_metrics (array of objects, MaxItems: 1024). Metrics to not send to AWS CloudWatch (takes precedence over extra_metrics). See below for nested schema.extra_metrics (array of objects, MaxItems: 1024). Metrics to allow through to AWS CloudWatch (in addition to default metrics). See below for nested schema.Appears on spec.externalAWSCloudwatchMetrics.
Metric name and subfield.
Required
field (string, MaxLength: 1000). Identifier of a value in the metric.metric (string, MaxLength: 1000). Identifier of the metric.Appears on spec.externalAWSCloudwatchMetrics.
Metric name and subfield.
Required
field (string, MaxLength: 1000). Identifier of a value in the metric.metric (string, MaxLength: 1000). Identifier of the metric.Appears on spec.
Kafka Connect service configuration values.
Required
kafka_connect (object). Kafka Connect service configuration values. See below for nested schema.Appears on spec.kafkaConnect.
Kafka Connect service configuration values.
Optional
config_storage_topic (string, MaxLength: 249). The name of the topic where connector and task configuration data are stored.This must be the same for all workers with the same group_id.group_id (string, MaxLength: 249). A unique string that identifies the Connect cluster group this worker belongs to.offset_storage_topic (string, MaxLength: 249). The name of the topic where connector and task configuration offsets are stored.This must be the same for all workers with the same group_id.status_storage_topic (string, MaxLength: 249). The name of the topic where connector and task configuration status updates are stored.This must be the same for all workers with the same group_id.Appears on spec.
Kafka logs configuration values.
Required
kafka_topic (string, MinLength: 1, MaxLength: 249). Topic name.Optional
selected_log_fields (array of strings, MaxItems: 5). The list of logging fields that will be sent to the integration logging service. The MESSAGE and timestamp fields are always sent.Appears on spec.
Kafka MirrorMaker configuration values.
Optional
cluster_alias (string, Pattern: ^[a-zA-Z0-9_.-]+$, MaxLength: 128). The alias under which the Kafka cluster is known to MirrorMaker. Can contain the following symbols: ASCII alphanumerics, ., _, and -.kafka_mirrormaker (object). Kafka MirrorMaker configuration values. See below for nested schema.Appears on spec.kafkaMirrormaker.
Kafka MirrorMaker configuration values.
Optional
consumer_auto_offset_reset (string, Enum: earliest, latest). Set where consumer starts to consume data. Value earliest: Start replication from the earliest offset. Value latest: Start replication from the latest offset. Default is earliest.consumer_fetch_min_bytes (integer, Minimum: 1, Maximum: 5242880). The minimum amount of data the server should return for a fetch request.consumer_max_poll_records (integer, Minimum: 100, Maximum: 20000). Set consumer max.poll.records. The default is 500.producer_batch_size (integer, Minimum: 0, Maximum: 5242880). The batch size in bytes producer will attempt to collect before publishing to broker.producer_buffer_memory (integer, Minimum: 5242880, Maximum: 134217728). The amount of bytes producer can use for buffering data before publishing to broker.producer_compression_type (string, Enum: gzip, snappy, lz4, zstd, none). Specify the default compression type for producers. This configuration accepts the standard compression codecs (gzip, snappy, lz4, zstd). It additionally accepts none which is the default and equivalent to no compression.producer_linger_ms (integer, Minimum: 0, Maximum: 5000). The linger time (ms) for waiting new data to arrive for publishing.producer_max_request_size (integer, Minimum: 0, Maximum: 268435456). The maximum request size in bytes.Appears on spec.
Logs configuration values.
Optional
elasticsearch_index_days_max (integer, Minimum: 1, Maximum: 10000). Elasticsearch index retention limit.elasticsearch_index_prefix (string, MinLength: 1, MaxLength: 1024). Elasticsearch index prefix.selected_log_fields (array of strings, MaxItems: 5). The list of logging fields that will be sent to the integration logging service. The MESSAGE and timestamp fields are always sent.Appears on spec.
Metrics configuration values.
Optional
database (string, Pattern: ^[_A-Za-z0-9][-_A-Za-z0-9]{0,39}$, MaxLength: 40). Name of the database where to store metric datapoints. Only affects PostgreSQL destinations. Defaults to metrics. Note that this must be the same for all metrics integrations that write data to the same PostgreSQL service.retention_days (integer, Minimum: 0, Maximum: 10000). Number of days to keep old metrics. Only affects PostgreSQL destinations. Set to 0 for no automatic cleanup. Defaults to 30 days.ro_username (string, Pattern: ^[_A-Za-z0-9][-._A-Za-z0-9]{0,39}$, MaxLength: 40). Name of a user that can be used to read metrics. This will be used for Grafana integration (if enabled) to prevent Grafana users from making undesired changes. Only affects PostgreSQL destinations. Defaults to metrics_reader. Note that this must be the same for all metrics integrations that write data to the same PostgreSQL service.source_mysql (object). Configuration options for metrics where source service is MySQL. See below for nested schema.username (string, Pattern: ^[_A-Za-z0-9][-._A-Za-z0-9]{0,39}$, MaxLength: 40). Name of the user used to write metrics. Only affects PostgreSQL destinations. Defaults to metrics_writer. Note that this must be the same for all metrics integrations that write data to the same PostgreSQL service.Appears on spec.metrics.
Configuration options for metrics where source service is MySQL.
Required
telegraf (object). Configuration options for Telegraf MySQL input plugin. See below for nested schema.Appears on spec.metrics.source_mysql.
Configuration options for Telegraf MySQL input plugin.
Optional
gather_event_waits (boolean). Gather metrics from PERFORMANCE_SCHEMA.EVENT_WAITS.gather_file_events_stats (boolean). gather metrics from PERFORMANCE_SCHEMA.FILE_SUMMARY_BY_EVENT_NAME.gather_index_io_waits (boolean). Gather metrics from PERFORMANCE_SCHEMA.TABLE_IO_WAITS_SUMMARY_BY_INDEX_USAGE.gather_info_schema_auto_inc (boolean). Gather auto_increment columns and max values from information schema.gather_innodb_metrics (boolean). Gather metrics from INFORMATION_SCHEMA.INNODB_METRICS.gather_perf_events_statements (boolean). Gather metrics from PERFORMANCE_SCHEMA.EVENTS_STATEMENTS_SUMMARY_BY_DIGEST.gather_process_list (boolean). Gather thread state counts from INFORMATION_SCHEMA.PROCESSLIST.gather_slave_status (boolean). Gather metrics from SHOW SLAVE STATUS command output.gather_table_io_waits (boolean). Gather metrics from PERFORMANCE_SCHEMA.TABLE_IO_WAITS_SUMMARY_BY_TABLE.gather_table_lock_waits (boolean). Gather metrics from PERFORMANCE_SCHEMA.TABLE_LOCK_WAITS.gather_table_schema (boolean). Gather metrics from INFORMATION_SCHEMA.TABLES.perf_events_statements_digest_text_limit (integer, Minimum: 1, Maximum: 2048). Truncates digest text from perf_events_statements into this many characters.perf_events_statements_limit (integer, Minimum: 1, Maximum: 4000). Limits metrics from perf_events_statements.perf_events_statements_time_limit (integer, Minimum: 1, Maximum: 2592000). Only include perf_events_statements whose last seen is less than this many seconds.apiVersion: aiven.io/v1alpha1\nkind: ServiceIntegrationEndpoint\nmetadata:\n name: my-service-integration-endpoint\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n project: aiven-project-name\n endpointName: my-external-postgresql\n endpointType: external_postgresql\n\n externalPostgresql:\n username: username\n password: password\n host: example.example\n port: 5432\n ssl_mode: require\napiVersion: aiven.io/v1alpha1\nkind: ServiceIntegrationEndpoint\nmetadata:\n name: my-service-integration-endpoint\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n project: aiven-project-name\n endpointName: my-external-schema-registry\n endpointType: external_schema_registry\n\n externalSchemaRegistry:\n url: https://schema-registry.example.com:8081\n authentication: basic\n basic_auth_username: username\n basic_auth_password: password\nInfo
To create this resource, a Secret containing Aiven token must be created first.
Apply the resource with:
kubectl apply -f example.yaml\nVerify the newly created ServiceIntegrationEndpoint:
kubectl get serviceintegrationendpoints my-service-integration-endpoint\nThe output is similar to the following:
Name Project Endpoint Name Endpoint Type ID \nmy-service-integration-endpoint aiven-project-name my-external-postgresql external_postgresql <id> \nServiceIntegrationEndpoint is the Schema for the serviceintegrationendpoints API.
Required
apiVersion (string). Value aiven.io/v1alpha1.kind (string). Value ServiceIntegrationEndpoint.metadata (object). Data that identifies the object, including a name string and optional namespace.spec (object). ServiceIntegrationEndpointSpec defines the desired state of ServiceIntegrationEndpoint. See below for nested schema.Appears on ServiceIntegrationEndpoint.
ServiceIntegrationEndpointSpec defines the desired state of ServiceIntegrationEndpoint.
Required
endpointType (string, Enum: autoscaler, datadog, external_aws_cloudwatch_logs, external_aws_cloudwatch_metrics, external_aws_s3, external_clickhouse, external_elasticsearch_logs, external_google_cloud_bigquery, external_google_cloud_logging, external_kafka, external_mysql, external_opensearch_logs, external_postgresql, external_redis, external_schema_registry, external_sumologic_logs, jolokia, prometheus, rsyslog, Immutable). Type of the service integration endpoint.project (string, Immutable, Pattern: ^[a-zA-Z0-9_-]+$, MaxLength: 63). Identifies the project this resource belongs to.Optional
authSecretRef (object). Authentication reference to Aiven token in a secret. See below for nested schema.datadog (object). Datadog configuration values. See below for nested schema.endpointName (string, Immutable, MaxLength: 36). Source endpoint for the integration (if any).externalAWSCloudwatchLogs (object). ExternalAwsCloudwatchLogs configuration values. See below for nested schema.externalAWSCloudwatchMetrics (object). ExternalAwsCloudwatchMetrics configuration values. See below for nested schema.externalElasticsearchLogs (object). ExternalElasticsearchLogs configuration values. See below for nested schema.externalGoogleCloudBigquery (object). ExternalGoogleCloudBigquery configuration values. See below for nested schema.externalGoogleCloudLogging (object). ExternalGoogleCloudLogging configuration values. See below for nested schema.externalKafka (object). ExternalKafka configuration values. See below for nested schema.externalOpensearchLogs (object). ExternalOpensearchLogs configuration values. See below for nested schema.externalPostgresql (object). ExternalPostgresql configuration values. See below for nested schema.externalSchemaRegistry (object). ExternalSchemaRegistry configuration values. See below for nested schema.jolokia (object). Jolokia configuration values. See below for nested schema.prometheus (object). Prometheus configuration values. See below for nested schema.rsyslog (object). Rsyslog configuration values. See below for nested schema.Appears on spec.
Authentication reference to Aiven token in a secret.
Required
key (string, MinLength: 1).name (string, MinLength: 1).Appears on spec.
Datadog configuration values.
Required
datadog_api_key (string, Pattern: ^[A-Za-z0-9]{1,256}$, MinLength: 1, MaxLength: 256). Datadog API key.Optional
datadog_tags (array of objects, MaxItems: 32). Custom tags provided by user. See below for nested schema.disable_consumer_stats (boolean). Disable consumer group metrics.kafka_consumer_check_instances (integer, Minimum: 1, Maximum: 100). Number of separate instances to fetch kafka consumer statistics with.kafka_consumer_stats_timeout (integer, Minimum: 2, Maximum: 300). Number of seconds that datadog will wait to get consumer statistics from brokers.max_partition_contexts (integer, Minimum: 200, Maximum: 200000). Maximum number of partition contexts to send.site (string, Enum: datadoghq.com, datadoghq.eu, us3.datadoghq.com, us5.datadoghq.com, ddog-gov.com, ap1.datadoghq.com). Datadog intake site. Defaults to datadoghq.com.Appears on spec.datadog.
Datadog tag defined by user.
Required
tag (string, MinLength: 1, MaxLength: 200). Tag format and usage are described here: https://docs.datadoghq.com/getting_started/tagging. Tags with prefix aiven- are reserved for Aiven.Optional
comment (string, MaxLength: 1024). Optional tag explanation.Appears on spec.
ExternalAwsCloudwatchLogs configuration values.
Required
access_key (string, MaxLength: 4096). AWS access key. Required permissions are logs:CreateLogGroup, logs:CreateLogStream, logs:PutLogEvents and logs:DescribeLogStreams.region (string, MaxLength: 32). AWS region.secret_key (string, MaxLength: 4096). AWS secret key.Optional
log_group_name (string, Pattern: ^[\\.\\-_/#A-Za-z0-9]+$, MinLength: 1, MaxLength: 512). AWS CloudWatch log group name.Appears on spec.
ExternalAwsCloudwatchMetrics configuration values.
Required
access_key (string, MaxLength: 4096). AWS access key. Required permissions are cloudwatch:PutMetricData.namespace (string, MinLength: 1, MaxLength: 255). AWS CloudWatch Metrics Namespace.region (string, MaxLength: 32). AWS region.secret_key (string, MaxLength: 4096). AWS secret key.Appears on spec.
ExternalElasticsearchLogs configuration values.
Required
index_prefix (string, Pattern: ^[a-z0-9][a-z0-9-_.]+$, MinLength: 1, MaxLength: 1000). Elasticsearch index prefix.url (string, MinLength: 12, MaxLength: 2048). Elasticsearch connection URL.Optional
ca (string, MaxLength: 16384). PEM encoded CA certificate.index_days_max (integer, Minimum: 1, Maximum: 10000). Maximum number of days of logs to keep.timeout (number, Minimum: 10, Maximum: 120). Elasticsearch request timeout limit.Appears on spec.
ExternalGoogleCloudBigquery configuration values.
Required
project_id (string, MinLength: 6, MaxLength: 30). GCP project id.service_account_credentials (string, MaxLength: 4096). This is a JSON object with the fields documented in https://cloud.google.com/iam/docs/creating-managing-service-account-keys .Appears on spec.
ExternalGoogleCloudLogging configuration values.
Required
log_id (string, MaxLength: 512). Google Cloud Logging log id.project_id (string, MinLength: 6, MaxLength: 30). GCP project id.service_account_credentials (string, MaxLength: 4096). This is a JSON object with the fields documented in https://cloud.google.com/iam/docs/creating-managing-service-account-keys .Appears on spec.
ExternalKafka configuration values.
Required
bootstrap_servers (string, MinLength: 3, MaxLength: 256). Bootstrap servers.security_protocol (string, Enum: PLAINTEXT, SSL, SASL_PLAINTEXT, SASL_SSL). Security protocol.Optional
sasl_mechanism (string, Enum: PLAIN, SCRAM-SHA-256, SCRAM-SHA-512). SASL mechanism used for connections to the Kafka server.sasl_plain_password (string, MinLength: 1, MaxLength: 256). Password for SASL PLAIN mechanism in the Kafka server.sasl_plain_username (string, MinLength: 1, MaxLength: 256). Username for SASL PLAIN mechanism in the Kafka server.ssl_ca_cert (string, MaxLength: 16384). PEM-encoded CA certificate.ssl_client_cert (string, MaxLength: 16384). PEM-encoded client certificate.ssl_client_key (string, MaxLength: 16384). PEM-encoded client key.ssl_endpoint_identification_algorithm (string, Enum: https). The endpoint identification algorithm to validate server hostname using server certificate.Appears on spec.
ExternalOpensearchLogs configuration values.
Required
index_prefix (string, Pattern: ^[a-z0-9][a-z0-9-_.]+$, MinLength: 1, MaxLength: 1000). OpenSearch index prefix.url (string, MinLength: 12, MaxLength: 2048). OpenSearch connection URL.Optional
ca (string, MaxLength: 16384). PEM encoded CA certificate.index_days_max (integer, Minimum: 1, Maximum: 10000). Maximum number of days of logs to keep.timeout (number, Minimum: 10, Maximum: 120). OpenSearch request timeout limit.Appears on spec.
ExternalPostgresql configuration values.
Required
host (string, MaxLength: 255). Hostname or IP address of the server.port (integer, Minimum: 1, Maximum: 65535). Port number of the server.username (string, MaxLength: 256). User name.Optional
default_database (string, Pattern: ^[_A-Za-z0-9][-_A-Za-z0-9]{0,62}$, MaxLength: 63). Default database.password (string, MaxLength: 256). Password.ssl_client_certificate (string, MaxLength: 16384). Client certificate.ssl_client_key (string, MaxLength: 16384). Client key.ssl_mode (string, Enum: require, verify-ca, verify-full). SSL mode to use for the connection. Please note that Aiven requires TLS for all connections to external PostgreSQL services.ssl_root_cert (string, MaxLength: 16384). SSL Root Cert.Appears on spec.
ExternalSchemaRegistry configuration values.
Required
authentication (string, Enum: none, basic). Authentication method.url (string, MaxLength: 2048). Schema Registry URL.Optional
basic_auth_password (string, MaxLength: 256). Basic authentication password.basic_auth_username (string, MaxLength: 256). Basic authentication user name.Appears on spec.
Jolokia configuration values.
Optional
basic_auth_password (string, MinLength: 8, MaxLength: 64). Jolokia basic authentication password.basic_auth_username (string, Pattern: ^[a-z0-9\\-@_]{5,32}$, MinLength: 5, MaxLength: 32). Jolokia basic authentication username.Appears on spec.
Prometheus configuration values.
Optional
basic_auth_password (string, MinLength: 8, MaxLength: 64). Prometheus basic authentication password.basic_auth_username (string, Pattern: ^[a-z0-9\\-@_]{5,32}$, MinLength: 5, MaxLength: 32). Prometheus basic authentication username.Appears on spec.
Rsyslog configuration values.
Required
format (string, Enum: rfc5424, rfc3164, custom). Message format.port (integer, Minimum: 1, Maximum: 65535). Rsyslog server port.server (string, MinLength: 4, MaxLength: 255). Rsyslog server IP address or hostname.tls (boolean). Require TLS.Optional
ca (string, MaxLength: 16384). PEM encoded CA certificate.cert (string, MaxLength: 16384). PEM encoded client certificate.key (string, MaxLength: 16384). PEM encoded client key.logline (string, Pattern: ^[ -~\\t]+$, MinLength: 1, MaxLength: 512). Custom syslog message format.max_message_size (integer, Minimum: 2048, Maximum: 2147483647). Rsyslog max message size.sd (string, MaxLength: 1024). Structured data block for log message.apiVersion: aiven.io/v1alpha1\nkind: ServiceUser\nmetadata:\n name: my-service-user\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n connInfoSecretTarget:\n name: service-user-secret\n prefix: MY_SECRET_PREFIX_\n annotations:\n foo: bar\n labels:\n baz: egg\n\n project: aiven-project-name\n serviceName: my-service-name\nInfo
To create this resource, a Secret containing Aiven token must be created first.
Apply the resource with:
kubectl apply -f example.yaml\nVerify the newly created ServiceUser:
kubectl get serviceusers my-service-user\nThe output is similar to the following:
Name Service Name Project \nmy-service-user my-service-name aiven-project-name \nTo view the details of the Secret, use the following command:
kubectl describe secret service-user-secret\nYou can use the jq to quickly decode the Secret:
kubectl get secret service-user-secret -o json | jq '.data | map_values(@base64d)'\nThe output is similar to the following:
{\n \"SERVICEUSER_HOST\": \"<secret>\",\n \"SERVICEUSER_PORT\": \"<secret>\",\n \"SERVICEUSER_USERNAME\": \"<secret>\",\n \"SERVICEUSER_PASSWORD\": \"<secret>\",\n \"SERVICEUSER_CA_CERT\": \"<secret>\",\n \"SERVICEUSER_ACCESS_CERT\": \"<secret>\",\n \"SERVICEUSER_ACCESS_KEY\": \"<secret>\",\n}\nServiceUser is the Schema for the serviceusers API.
Exposes secret keys
SERVICEUSER_HOST, SERVICEUSER_PORT, SERVICEUSER_USERNAME, SERVICEUSER_PASSWORD, SERVICEUSER_CA_CERT, SERVICEUSER_ACCESS_CERT, SERVICEUSER_ACCESS_KEY.
Required
apiVersion (string). Value aiven.io/v1alpha1.kind (string). Value ServiceUser.metadata (object). Data that identifies the object, including a name string and optional namespace.spec (object). ServiceUserSpec defines the desired state of ServiceUser. See below for nested schema.Appears on ServiceUser.
ServiceUserSpec defines the desired state of ServiceUser.
Required
project (string, Immutable, Pattern: ^[a-zA-Z0-9_-]+$, MaxLength: 63). Identifies the project this resource belongs to.serviceName (string, Immutable, Pattern: ^[a-z][-a-z0-9]+$, MaxLength: 63). Specifies the name of the service that this resource belongs to.Optional
authSecretRef (object). Authentication reference to Aiven token in a secret. See below for nested schema.authentication (string, Enum: caching_sha2_password, mysql_native_password). Authentication details.connInfoSecretTarget (object). Secret configuration. See below for nested schema.connInfoSecretTargetDisabled (boolean, Immutable). When true, the secret containing connection information will not be created, defaults to false. This field cannot be changed after resource creation.Appears on spec.
Authentication reference to Aiven token in a secret.
Required
key (string, MinLength: 1).name (string, MinLength: 1).Appears on spec.
Secret configuration.
Required
name (string, Immutable). Name of the secret resource to be created. By default, it is equal to the resource name.Optional
annotations (object, AdditionalProperties: string). Annotations added to the secret.labels (object, AdditionalProperties: string). Labels added to the secret.prefix (string). Prefix for the secret's keys. Added \"as is\" without any transformations. By default, is equal to the kind name in uppercase + underscore, e.g. KAFKA_, REDIS_, etc.The Aiven Operator for Kubernetes project accepts contributions via GitHub pull requests. This document outlines the process to help get your contribution accepted.
Please see also the Aiven Operator for Kubernetes Developer Guide.
"},{"location":"contributing/index.html#support-channels","title":"Support Channels","text":"This project offers support through GitHub issues and can be filed here. Moreover, GitHub issues are used as the primary method for tracking anything to do with the Aiven Operator for Kubernetes project.
"},{"location":"contributing/index.html#pull-request-process","title":"Pull Request Process","text":"In the interest of fostering an open and welcoming environment, we as contributors and maintainers pledge to making participation in our project and our community a harassment-free experience for everyone, regardless of age, body size, disability, ethnicity, gender identity and expression, level of experience, nationality, personal appearance, race, religion, or sexual identity and orientation.
"},{"location":"contributing/index.html#our-standards","title":"Our Standards","text":"Examples of behavior that contributes to creating a positive environment include:
Examples of unacceptable behavior by participants include:
This project adheres to the Conventional Commits specification. Please, make sure that your commit messages follow that specification.
"},{"location":"contributing/index.html#our-responsibilities","title":"Our Responsibilities","text":"Project maintainers are responsible for clarifying the standards of acceptable behavior and are expected to take appropriate and fair corrective action in response to any instances of unacceptable behavior.
Project maintainers have the right and responsibility to remove, edit, or reject comments, commits, code, wiki edits, issues, and other contributions that are not aligned to this Code of Conduct, or to ban temporarily or permanently any contributor for other behaviors that they deem inappropriate, threatening, offensive, or harmful.
"},{"location":"contributing/index.html#scope","title":"Scope","text":"This Code of Conduct applies both within project spaces and in public spaces when an individual is representing the project or its community. Examples of representing a project or community include using an official project e-mail address, posting via an official social media account, or acting as an appointed representative at an online or offline event. Representation of a project may be further defined and clarified by project maintainers.
"},{"location":"contributing/index.html#enforcement","title":"Enforcement","text":"Instances of abusive, harassing, or otherwise unacceptable behavior may be reported by contacting the project team at opensource@aiven.io. All complaints will be reviewed and investigated and will result in a response that is deemed necessary and appropriate to the circumstances. The project team is obligated to maintain confidentiality with regard to the reporter of an incident. Further details of specific enforcement policies may be posted separately.
Project maintainers who do not follow or enforce the Code of Conduct in good faith may face temporary or permanent repercussions as determined by other members of the project's leadership.
"},{"location":"contributing/developer-guide.html","title":"Developer guide","text":""},{"location":"contributing/developer-guide.html#get-started","title":"Get Started","text":"To get started, make sure you have a working Go environment and clone the repository:
git clone git@github.com:aiven/aiven-operator.git\ncd aiven-operator\nBefore proceeding, please ensure that you have the following dependencies installed:
-w0 flag, which may cause some tests to not work properly)Create an Aiven authentication token and an Aiven project, you'll need them in the following sections:
AIVEN_TOKEN \u2014 your authentication tokenAIVEN_PROJECT_NAME \u2014 the name of you Aiven project to run the services inOptional dependencies:
The project uses the make build system. To see the available targets, run make help.
Building the operator binary:
make build\nTo run the operator locally, first create a local k8s cluster with kind:
$ kind create cluster --image kindest/node:v1.26.6 --wait 5m\nCreating cluster \"kind\" ...\n \u2713 Ensuring node image (kindest/node:v1.26.6) \ud83d\uddbc\n \u2713 Preparing nodes \ud83d\udce6\n \u2713 Writing configuration \ud83d\udcdc\n \u2713 Starting control-plane \ud83d\udd79\ufe0f\n \u2713 Installing CNI \ud83d\udd0c\n \u2713 Installing StorageClass \ud83d\udcbe\n \u2713 Waiting \u2264 5m0s for control-plane = Ready \u23f3\n \u2022 Ready after 14s \ud83d\udc9a\nSet kubectl context to \"kind-kind\"\nYou can now use your cluster with:\n\nkubectl cluster-info --context kind-kinds\nThis creates a k8s cluster and points your kubectl to that cluster:
$ kubectl cluster-info --context kind-kind\nKubernetes control plane is running at https://127.0.0.1:58083\nCoreDNS is running at https://127.0.0.1:58083/api/v1/namespaces/kube-system/services/kube-dns:dns/proxy\nRun the operator:
ENABLE_WEBHOOKS=false make run\nIn another terminal window, create a Kubernetes Secret named aiven-token (use the Aiven authentication token created earlier):
export AIVEN_TOKEN=...\nkubectl create secret generic aiven-token --from-literal=token=$AIVEN_TOKEN\nThen, create a manifest for an Aiven PostgreSQL service:
aiven-pg.yamlapiVersion: aiven.io/v1alpha1\nkind: PostgreSQL\nmetadata:\n name: aiven-devdocs-pg\nspec:\n # reads the authentication token\n authSecretRef:\n name: aiven-token\n key: token\n\n # stores the PostgreSQL connection information on the specified Secret\n connInfoSecretTarget:\n name: pg-connection\n\n project: <your-project-name>\n cloudName: google-europe-west1\n plan: hobbyist\n maintenanceWindowDow: friday\n maintenanceWindowTime: 23:00:00\n userConfig:\n pg_version: \"15\"\n...and create the service at Aiven by applying the manifest:
kubectl apply -f aiven-pg.yaml\nYou should see the service be created on Aiven's side using either Aiven Console or Aiven CLI:
$ avn project switch PROJECT_NAME\n$ avn service list\nSERVICE_NAME SERVICE_TYPE STATE CLOUD_NAME PLAN CREATE_TIME UPDATE_TIME NOTIFICATIONS\n================ ============ ========== =================== ======== ==================== ==================== =============\naiven-devdocs-pg pg REBUILDING google-europe-west1 hobbyist 2024-01-01T00:00:00Z 2024-01-01T00:01:00Z\nTo see the resource on k8s side, use kubectl:
$ kubectl get postgresqls\nNAME PROJECT REGION PLAN STATE\naiven-devdocs-pg aiven-sbox-k8s-riski google-europe-west1 hobbyist REBUILDING\n... or by running k9s in another terminal window and type \": postgresqls\" to see the provisioned PostgreSQL resources:
For this section you'll need to set export AIVEN_TOKEN=... and an export AIVEN_PROJECT_NAME=... as well as export ENABLE_WEBHOOKS=false (see Prerequisites).
Note
If you encounter any issues with webhooks, you have the option to disable them by running export ENABLE_WEBOOKS=false
Running all unit tests is possible by running make test. To run a targeted test, use run=TestGrafana make test.
Setup the test environment:
make e2e-setup-kind\nThe e2e tests are configured to run against a project called aiven-ci-kubernetes-operator. To run the test against
make test-e2e-preinstalled\nTo remove the kind cluster, simply execute the following command:
kind delete cluster\nPlease see this page for more information.
"},{"location":"contributing/developer-guide.html#documentation","title":"Documentation","text":"The documentation is written in markdown and generated by mkdocs and mkdocs-material.
To run the documentation live preview:
make serve-docs\nAnd open the http://localhost:8000/aiven-operator/ page in your web browser.
The documentation API Reference section is generated automatically from the source code during the documentation deployment. To generate it locally, run the following command:
make docs\nAiven Kubernetes Operator generates service configs code (also known as user configs) and documentation from public service types schema.
"},{"location":"contributing/resource-generation.html#flow-overview","title":"Flow overview","text":"When a new schema is issued on the API, a cron job fetches it, parses, patches, and saves in a shared library \u2014 go-api-schemas.
When the library is updated, the GitHub dependabot creates PRs to the dependent repositories, like Aiven Kubernetes Operator and Aiven Terraform Provider.
Then the make generate command is called by GitHub action. And the PR is ready for review.
flowchart TB\n API(Aiven API) <-.->|polls schema updates| Schema([go-api-schemas])\n Bot(dependabot) <-.->|polls updates| Schema\n Bot-->|pull request|UpdateOP[/\"\u2728 $ make generate \u2728\"/]\n UpdateOP-->|review| OP([operator repository])The command runs several generators in a certain sequence. First, the user config generator is called. Then controller-gen cli. Then API reference docs generator and charts generator.
Here how it goes in the details:
./<api-reference-docs>/example/, if it finds one, it validates that with the CRD. Each CRD has an OpenAPI v3 schema as a part of it. This is also used by Kubernetes itself to validate user input.flowchart TB\n Make[/$ make generate/]-->Generator(userconfig generator<br> creates/updates structs using updated spec)\n Generator-->|go: KafkaUserConfig struct| K8S(controller-gen<br> adds k8s methods to structs)\n K8S-->|go files| CRD(controller-gen<br> creates CRDs out of structs)\n CRD-->|CRD: aiven.io_kafkas.yaml| Docs(docs generator)\n subgraph API reference generation\n Docs-->|aiven.io_kafkas.yaml|Reference(creates reference<br> out of CRD)\n Docs-->|examples/kafka.yaml,<br> aiven.io_kafkas.yaml|Examples(validates example<br> using CRD)\n Examples--> Markdown(creates docs out of CRDs, adds examples)\n Reference-->Markdown(kafka.md)\n end\n CRD-->|yaml files|Charts(charts generator<br> updates helm charts<br> and the changelog)\n Charts-->ToRelease(\"Ready to release \ud83c\udf89\")\n Markdown-->ToReleaseBy default, charts generator keeps the current helm chart's version, because it doesn't know semver. You need it to do manually.
To do so run the following command with the version of your choice:
make version=v1.0.0 charts\nThe Aiven Operator for Kubernetes can be installed via Helm.
Before you start, make sure you have the prerequisites.
First add the Aiven Helm repository:
helm repo add aiven https://aiven.github.io/aiven-charts && helm repo update\nhelm install aiven-operator-crds aiven/aiven-operator-crds\nVerify the installation:
kubectl api-resources --api-group=aiven.io\nThe output is similar to the following:
NAME SHORTNAMES APIVERSION NAMESPACED KIND\nconnectionpools aiven.io/v1alpha1 true ConnectionPool\ndatabases aiven.io/v1alpha1 true Database\n... < several omitted lines >\nhelm install aiven-operator aiven/aiven-operator\nNote
Installation will fail if webhooks are enabled and the CRDs for the cert-manager are not installed.
Verify the installation:
helm status aiven-operator\nThe output is similar to the following:
NAME: aiven-operator\nLAST DEPLOYED: Fri Sep 10 15:23:26 2021\nNAMESPACE: default\nSTATUS: deployed\nREVISION: 1\nTEST SUITE: None\nIt is also possible to install the operator without webhooks enabled:
helm install aiven-operator aiven/aiven-operator --set webhooks.enabled=false\nPlease refer to the values.yaml of the chart.
"},{"location":"installation/helm.html#install-without-full-cluster-administrator-access","title":"Install without full cluster administrator access","text":"If the person installing the Helm chart does not have the necessary permissions to create cluster-wide resources such as ClusterRole and ClusterRoleBinding, a cluster administrator can manually install these roles. This ensures that the operator can function properly.
Important
Please see this page for more information.
Find out the name of your deployment:
helm list\nThe output has the name of each deployment similar to the following:
NAME NAMESPACE REVISION UPDATED STATUS CHART APP VERSION\naiven-operator default 1 2021-09-09 10:56:14.623700249 +0200 CEST deployed aiven-operator-v0.1.0 v0.1.0\naiven-operator-crds default 1 2021-09-09 10:56:05.736411868 +0200 CEST deployed aiven-operator-crds-v0.1.0 v0.1.0\nRemove the CRDs:
helm uninstall aiven-operator-crds\nThe confirmation message is similar to the following:
release \"aiven-operator-crds\" uninstalled\nRemove the operator:
helm uninstall aiven-operator\nThe confirmation message is similar to the following:
release \"aiven-operator\" uninstalled\nBefore you start, make sure you have the prerequisites.
All Aiven Operator for Kubernetes components can be installed from one YAML file that is uploaded for every release:
kubectl apply -f https://github.com/aiven/aiven-operator/releases/latest/download/deployment.yaml\nBy default the Deployment is installed into the aiven-operator-system namespace.
Assuming you installed version vX.Y.Z of the operator it can be uninstalled via
kubectl delete -f https://github.com/aiven/aiven-operator/releases/download/vX.Y.Z/deployment.yaml\nThe Aiven Operator for Kubernetes supports all major Kubernetes distributions, both locally and in the cloud.
Make sure you have the following:
The Aiven Operator for Kubernetes uses cert-manager to configure the service reference of our webhooks.
Please follow the installation instructions on their website.
Note
This is not required in the Helm installation if you select to disable webhooks, but that is not recommended outside of playground use.
The Aiven Operator for Kubernetes uses webhooks for setting defaults and enforcing invariants that are expected by the aiven API and will lead to errors if ignored.
In the future webhooks will also be used for conversion and supporting multiple CRD versions.
"},{"location":"installation/uninstalling.html","title":"Uninstall","text":"Danger
Uninstalling the Aiven Operator for Kubernetes can remove the resources created in Aiven, possibly resulting in data loss.
Depending on your installation, please follow one of:
Aiven resources need to have an accompanying secret that contains the token that is used to authorize the manipulation of that resource. If that token expired then you will not be able to delete the custom resource and deletion will also hang until the situation is resolved. The recommended approach to deal with that situation is to patch a valid token into the secret again so that proper cleanup of aiven resources can take place.
"},{"location":"installation/uninstalling.html#hanging-deletions","title":"Hanging deletions","text":"To protect the Secrets that the operator is using from deletion, it adds the finalizer finalizers.aiven.io/needed-to-delete-services to the Secret. This solves a race condition that happens when deleting a namespace, where there is a possibility of the Secret getting deleted before the resource that uses it. When the controller is deleted it may not cleanup the finalizers from all Secrets. If there is a Secret with this finalizer blocking deletion of a namespace, you can remove the finalizer by running:
kubectl patch secret <offending-secret> -p '{\"metadata\":{\"finalizers\":null}}' --type=merge\nAiven for Apache Cassandra\u00ae is a distributed database designed to handle large volumes of writes.
Note
Before going through this guide, make sure you have a Kubernetes cluster with the operator installed (see instructions for helm or kubectl), and a Kubernetes Secret with an Aiven authentication token.
"},{"location":"resources/cassandra.html#create-a-cassandra-instance","title":"Create a Cassandra instance","text":"1. Create a file named cassandra-sample.yaml, and add the following content:
apiVersion: aiven.io/v1alpha1\nkind: Cassandra\nmetadata:\n name: cassandra-sample\nspec:\n # gets the authentication token from the `aiven-token` Secret\n authSecretRef:\n name: aiven-token\n key: token\n\n # outputs the Cassandra connection on the `cassandra-secret` Secret\n connInfoSecretTarget:\n name: cassandra-secret\n\n # add your Project name here\n project: PROJECT_NAME\n\n # cloud provider and plan of your choice\n # you can check all of the possibilities here https://aiven.io/pricing\n cloudName: google-europe-west1\n plan: startup-4\n\n # general Aiven configuration\n maintenanceWindowDow: friday\n maintenanceWindowTime: 23:00:00\n2. Create the service by applying the configuration:
kubectl apply -f cassandra-sample.yaml\nThe output is:
cassandra.aiven.io/cassandra-sample created\n3. Review the resource you created with this command:
kubectl describe cassandra.aiven.io cassandra-sample\nThe output is similar to the following:
...\nStatus:\n Conditions:\n Last Transition Time: 2023-01-31T10:17:25Z\n Message: Successfully created or updated the instance in Aiven\n Reason: Created\n Status: True\n Type: Initialized\n Last Transition Time: 2023-01-31T10:24:00Z\n Message: Instance is running on Aiven side\n Reason: CheckRunning\n Status: True\n Type: Running\n State: RUNNING\n...\nThe resource can be in the REBUILDING state for a few minutes. Once the state changes to RUNNING, you can access the resource.
For your convenience, the operator automatically stores the Cassandra connection information in a Secret created with the name specified on the connInfoSecretTarget field.
To view the details of the Secret, use the following command:
kubectl describe secret cassandra-secret\nThe output is similar to the following:
Name: cassandra-secret\nNamespace: default\nLabels: <none>\nAnnotations: <none>\n\nType: Opaque\n\nData\n====\nCASSANDRA_HOSTS: 59 bytes\nCASSANDRA_PASSWORD: 24 bytes\nCASSANDRA_PORT: 5 bytes\nCASSANDRA_URI: 66 bytes\nCASSANDRA_USER: 8 bytes\nCASSANDRA_HOST: 60 bytes\nYou can use the jq to quickly decode the Secret:
kubectl get secret cassandra-secret -o json | jq '.data | map_values(@base64d)'\nThe output is similar to the following:
{\n \"CASSANDRA_HOST\": \"<secret>\",\n \"CASSANDRA_HOSTS\": \"<secret>\",\n \"CASSANDRA_PASSWORD\": \"<secret>\",\n \"CASSANDRA_PORT\": \"14609\",\n \"CASSANDRA_URI\": \"<secret>\",\n \"CASSANDRA_USER\": \"avnadmin\"\n}\nYou can create service users for your instance of Aiven for Apache Cassandra. Service users are unique to this instance and are not shared with any other services.
1. Create a file named cassandra-service-user.yaml:
apiVersion: aiven.io/v1alpha1\nkind: ServiceUser\nmetadata:\n name: cassandra-service-user\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n connInfoSecretTarget:\n name: cassandra-service-user-secret\n\n project: PROJECT_NAME\n serviceName: cassandra-sample\n2. Create the user by applying the configuration:
kubectl apply -f cassandra-service-user.yaml\nThe ServiceUser resource generates a Secret with connection information.
3. View the details of the Secret using the following command:
kubectl get secret cassandra-service-user-secret -o json | jq '.data | map_values(@base64d)'\nThe output is similar to the following:
{\n \"ACCESS_CERT\": \"<secret>\",\n \"ACCESS_KEY\": \"<secret>\",\n \"CA_CERT\": \"<secret>\",\n \"HOST\": \"<secret>\",\n \"PASSWORD\": \"<secret>\",\n \"PORT\": \"14609\",\n \"USERNAME\": \"cassandra-service-user\"\n}\nYou can connect to the Cassandra instance using these credentials and the host information from the cassandra-secret Secret.
Aiven for ClickHouse\u00ae is a fully managed distributed columnar database based on open source ClickHouse.
Note
Before going through this guide, make sure you have a Kubernetes cluster with the operator installed (see instructions for helm or kubectl), and a Kubernetes Secret with an Aiven authentication token.
"},{"location":"resources/clickhouse.html#create-a-clickhouse-instance","title":"Create a ClickHouse instance","text":"1. Create a file named clickhouse-sample.yaml, and add the following:
apiVersion: aiven.io/v1alpha1\nkind: Clickhouse\nmetadata:\n name: example-clickhouse\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n # Outputs the ClickHouse connection to the `clickhouse-secret` Secret.\n connInfoSecretTarget:\n name: clickhouse-secret\n # Your Aiven project.\n project: PROJECT_NAME\n # Choose a cloud provider and plan.\n # View the options on the pricing page at https://aiven.io/pricing.\n cloudName: google-europe-west1\n plan: startup-16\n # Configure the maintenance window.\n maintenanceWindowDow: friday\n maintenanceWindowTime: 23:00:00\nPROJECT_NAME is the name of your Aiven project. 2. To create the service, run:
kubectl apply -f clickhouse-sample.yaml\nThe output is similar to the following:
clickhouse.aiven.io/example-clickhouse created\n3. To review the resource you created, run:
kubectl describe clickhouse.aiven.io example-clickhouse\nThe output is similar to the following:
...\nStatus:\n Conditions:\n Last Transition Time: 2024-06-25T07:58:37Z\n Message: Instance was created or update on Aiven side\n Reason: Created\n Status: True\n Type: Initialized\n Last Transition Time: 2024-06-25T08:01:47Z\n Message: Instance is running on Aiven side\n Reason: CheckRunning\n Status: True\n Type: Running\n State: RUNNING\nEvents:\n Type Reason Age From Message\n ---- ------ ---- ---- -------\n Normal CreatedOrUpdatedAtAiven 3m24s (x5 over 3m45s) clickhouse-reconciler waiting for the instance to be running\n Normal ReconcilationStarted 3m14s (x6 over 3m47s) clickhouse-reconciler starting reconciliation\n Normal InstanceFinalizerAdded 3m14s (x6 over 3m47s) clickhouse-reconciler waiting for preconditions of the instance\n...\nREBUILDING state for a few minutes. Once the state changes to RUNNING, you can access the service."},{"location":"resources/clickhouse.html#use-the-connection-secret","title":"Use the connection Secret","text":"Aiven Operator automatically stores the ClickHouse connection information in a Secret created with the name in the connInfoSecretTarget field.
To view the details of the Secret, run:
kubectl describe secret clickhouse-secret\nName: clickhouse-secret\nNamespace: default\nLabels: <none>\nAnnotations: <none>\n\nType: Opaque\n\nData\n====\nCLICKHOUSE_USER: 8 bytes\nHOST: 46 bytes\nPASSWORD: 24 bytes\nPORT: 5 bytes\nUSER: 8 bytes\nCLICKHOUSE_HOST: 46 bytes\nCLICKHOUSE_PASSWORD: 24 bytes\nCLICKHOUSE_PORT: 5 bytes\nYou can use the JSON processor jq to decode the Secret:
kubectl get secret clickhouse-secret -o json | jq '.data | map_values(@base64d)'\nThe output is similar to the following:
{\n \"CLICKHOUSE_HOST\": \"HOST\",\n \"CLICKHOUSE_PASSWORD\": \"SERVICE_PASSWORD\",\n \"CLICKHOUSE_PORT\": \"12691\",\n \"CLICKHOUSE_USER\": \"avnadmin\",\n \"HOST\": \"HOST\",\n \"PASSWORD\": \"ADMIN_USER_PASSWORD\",\n \"PORT\": \"12691\",\n \"USER\": \"avnadmin\"\n}\nNote
Tables cannot be created using Aiven Operator. To create a table, use the Aiven Console or CLI.
1. Create a file named clickhouse-db.yaml and add the following:
apiVersion: aiven.io/v1alpha1\nkind: ClickhouseDatabase\nmetadata:\n name: example-database\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n serviceName: example-clickhouse\n project: PROJECT_NAME\nWhere PROJECT_NAME is the name of your Aiven project.
2. To create the database, run:
kubectl apply -f clickhouse-db.yaml\n3. To view details about the resource, run:
kubectl describe ClickhouseDatabase example-database\nThe output is similar to the following:
...\nSpec:\n Auth Secret Ref:\n Key: token\n Name: aiven-token\n Project: example-project\n Service Name: example-clickhouse\nStatus:\n Conditions:\n Last Transition Time: 2024-06-25T13:58:35Z\n Message: Checking preconditions\n Reason: Preconditions\n Status: True\n Type: Initialized\n Last Transition Time: 2024-06-25T14:32:05Z\n Message: Instance is running on Aiven side\n Reason: CheckRunning\n Status: True\n Type: Running\n...\nYou can create service users and roles for an instance of ClickHouse, and grant privileges to them. Users and roles are not shared with any other services.
1. Create a file named clickhouse-service-users.yaml and add the following configuration for a service user:
apiVersion: aiven.io/v1alpha1\nkind: ClickhouseUser\nmetadata:\n name: example-user\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n connInfoSecretTarget:\n name: clickhouse-service-user-secret\n serviceName: example-clickhouse\n project: PROJECT_NAME\nPROJECT_NAME is the name of your Aiven project. This resource generates a Secret with connection information and stores it in clickhouse-service-user-secret.
2. To create a role add the following to the same file:
---\n\napiVersion: aiven.io/v1alpha1\nkind: ClickhouseRole\nmetadata:\n name: example-role\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n serviceName: example-clickhouse\n project: PROJECT_NAME\n role: read-only\n3. Grant privileges to the role, and assign the role to the user by adding the following to the same file:
---\n\napiVersion: aiven.io/v1alpha1\nkind: ClickhouseGrant\nmetadata:\n name: example-grant\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n project: PROJECT_NAME\n serviceName: example-clickhouse\n\n privilegeGrants:\n - grantees:\n - role: read-only\n privileges:\n - SELECT\n database: example-database\n\n roleGrants:\n - roles:\n - read-only\n grantees:\n - user: example-user\n4. To create the user, role, and grant, run:
kubectl apply -f clickhouse-service-users.yaml\n5. To get credentials and host information for connecting to this instance, view the connection information stored in the Secret by running:
kubectl get secret clickhouse-service-user-secret -o json | jq '.data | map_values(@base64d)'\nThe output is similar to the following:
{\n \"CLICKHOUSEUSER_HOST\": \"HOST\",\n \"CLICKHOUSEUSER_PASSWORD\": \"PASSWORD\",\n \"CLICKHOUSEUSER_PORT\": \"12691\",\n \"CLICKHOUSEUSER_USERNAME\": \"example-user\",\n \"HOST\": \"HOST\",\n \"PASSWORD\": \"PASSWORD\",\n \"PORT\": \"12691\",\n \"USERNAME\": \"example-user\"\n}\nAiven for MySQL is a fully managed relational database service, deployable in the cloud of your choice.
Before going through this guide, make sure you have a Kubernetes cluster with the operator installed (see instructions for helm or kubectl) and a Kubernetes Secret with an Aiven authentication token.
"},{"location":"resources/mysql.html#create-a-mysql-instance","title":"Create a MySQL instance","text":"1. Create a file named mysql-sample.yaml, and add the following content:
apiVersion: aiven.io/v1alpha1\nkind: MySQL\nmetadata:\n name: mysql-sample\nspec:\n # gets the authentication token from the `aiven-token` Secret\n authSecretRef:\n name: aiven-token\n key: token\n\n # outputs the MySQL connection on the `mysql-secret` Secret\n connInfoSecretTarget:\n name: mysql-secret\n\n # add your Project name here\n project: PROJECT_NAME\n\n # cloud provider and plan of your choice\n # you can check all of the possibilities here https://aiven.io/pricing\n cloudName: google-europe-west1\n plan: startup-4\n\n # general Aiven configuration\n maintenanceWindowDow: friday\n maintenanceWindowTime: 23:00:00\n2. Create the service by applying the configuration:
kubectl apply -f mysql-sample.yaml\n3. Review the resource you created with this command:
kubectl describe mysql.aiven.io mysql-sample\nThe output is similar to the following:
...\nStatus:\n Conditions:\n Last Transition Time: 2023-02-22T15:43:44Z\n Message: Successfully created or updated the instance in Aiven\n Reason: Created\n Status: True\n Type: Initialized\n Last Transition Time: 2023-02-22T15:43:44Z\n Message: Successfully created or updated the instance in Aiven, status remains unknown\n Reason: Created\n Status: Unknown\n Type: Running\n State: REBUILDING\n...\nThe resource will be in the REBUILDING state for a few minutes. Once the state changes to RUNNING, you can access the resource.
For your convenience, the operator automatically stores the MySQL connection information in a Secret created with the name specified on the connInfoSecretTarget field.
To view the details of the Secret, use the following command:
kubectl describe secret mysql-secret\nThe output is similar to the following:
Name: mysql-secret\nNamespace: default\nLabels: <none>\nAnnotations: <none>\n\nType: Opaque\n\nData\n====\nMYSQL_PORT: 5 bytes\nMYSQL_SSL_MODE: 8 bytes\nMYSQL_URI: 115 bytes\nMYSQL_USER: 8 bytes\nMYSQL_DATABASE: 9 bytes\nMYSQL_HOST: 39 bytes\nMYSQL_PASSWORD: 24 bytes\nYou can use jq to quickly decode the Secret:
kubectl get secret mysql-secret -o json | jq '.data | map_values(@base64d)'\nThe output is similar to the following:
{\n \"MYSQL_DATABASE\": \"defaultdb\",\n \"MYSQL_HOST\": \"<secret>\",\n \"MYSQL_PASSWORD\": \"<secret>\",\n \"MYSQL_PORT\": \"12691\",\n \"MYSQL_SSL_MODE\": \"REQUIRED\",\n \"MYSQL_URI\": \"<secret>\",\n \"MYSQL_USER\": \"avnadmin\"\n}\nYou can create service users for your instance of Aiven for MySQL. Service users are unique to this instance and are not shared with any other services.
1. Create a file named mysql-service-user.yaml:
apiVersion: aiven.io/v1alpha1\nkind: ServiceUser\nmetadata:\n name: mysql-service-user\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n connInfoSecretTarget:\n name: mysql-service-user-secret\n\n project: PROJECT_NAME\n serviceName: mysql-sample\n2. Create the user by applying the configuration:
kubectl apply -f mysql-service-user.yaml\nThe ServiceUser resource generates a Secret with connection information.
3. View the details of the Secret using jq:
kubectl get secret mysql-service-user-secret -o json | jq '.data | map_values(@base64d)'\nThe output is similar to the following:
{\n \"ACCESS_CERT\": \"<secret>\",\n \"ACCESS_KEY\": \"<secret>\",\n \"CA_CERT\": \"<secret>\",\n \"HOST\": \"<secret>\",\n \"PASSWORD\": \"<secret>\",\n \"PORT\": \"14609\",\n \"USERNAME\": \"mysql-service-user\"\n}\nYou can connect to the MySQL instance using these credentials and the host information from the mysql-secret Secret.
OpenSearch\u00ae is an open source search and analytics suite including search engine, NoSQL document database, and visualization interface. OpenSearch offers a distributed, full-text search engine based on Apache Lucene\u00ae with a RESTful API interface and support for JSON documents.
Note
Before going through this guide, make sure you have a Kubernetes cluster with the operator installed (see instructions for helm or kubectl), and a Kubernetes Secret with an Aiven authentication token.
"},{"location":"resources/opensearch.html#create-an-opensearch-instance","title":"Create an OpenSearch instance","text":"1. Create a file named os-sample.yaml, and add the following content:
apiVersion: aiven.io/v1alpha1\nkind: OpenSearch\nmetadata:\n name: os-sample\nspec:\n # gets the authentication token from the `aiven-token` Secret\n authSecretRef:\n name: aiven-token\n key: token\n\n # outputs the OpenSearch connection on the `os-secret` Secret\n connInfoSecretTarget:\n name: os-secret\n\n # add your Project name here\n project: PROJECT_NAME\n\n # cloud provider and plan of your choice\n # you can check all of the possibilities here https://aiven.io/pricing\n cloudName: google-europe-west1\n plan: startup-4\n\n # general Aiven configuration\n maintenanceWindowDow: friday\n maintenanceWindowTime: 23:00:00\n2. Create the service by applying the configuration:
kubectl apply -f os-sample.yaml\n3. Review the resource you created with this command:
kubectl describe opensearch.aiven.io os-sample\nThe output is similar to the following:
...\nStatus:\n Conditions:\n Last Transition Time: 2023-01-19T14:41:43Z\n Message: Successfully created or updated the instance in Aiven\n Reason: Created\n Status: True\n Type: Initialized\n Last Transition Time: 2023-01-19T14:41:43Z\n Message: Successfully created or updated the instance in Aiven, status remains unknown\n Reason: Created\n Status: Unknown\n Type: Running\n State: REBUILDING\n...\nThe resource will be in the REBUILDING state for a few minutes. Once the state changes to RUNNING, you can access the resource.
For your convenience, the operator automatically stores the OpenSearch connection information in a Secret created with the name specified on the connInfoSecretTarget field.
To view the details of the Secret, use the following command:
kubectl describe secret os-secret\nThe output is similar to the following:
Name: os-secret\nNamespace: default\nLabels: <none>\nAnnotations: <none>\n\nType: Opaque\n\nData\n====\nHOST: 61 bytes\nPASSWORD: 24 bytes\nPORT: 5 bytes\nUSER: 8 bytes\nYou can use the jq to quickly decode the Secret:
kubectl get secret os-secret -o json | jq '.data | map_values(@base64d)'\nThe output is similar to the following:
{\n \"HOST\": \"os-sample-your-project.aivencloud.com\",\n \"PASSWORD\": \"<secret>\",\n \"PORT\": \"13041\",\n \"USER\": \"avnadmin\"\n}\nYou can create service users for your instance of Aiven for OpenSearch. Service users are unique to this instance and are not shared with any other services.
1. Create a file named os-service-user.yaml:
apiVersion: aiven.io/v1alpha1\nkind: ServiceUser\nmetadata:\n name: os-service-user\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n connInfoSecretTarget:\n name: os-service-user-secret\n\n project: PROJECT_NAME\n serviceName: os-sample\n2. Create the user by applying the configuration:
kubectl apply -f os-service-user.yaml\nThe ServiceUser resource generates a Secret with connection information.
3. View the details of the Secret using the following command:
kubectl get secret os-service-user-secret -o json | jq '.data | map_values(@base64d)'\nThe output is similar to the following:
{\n \"ACCESS_CERT\": \"<secret>\",\n \"ACCESS_KEY\": \"<secret>\",\n \"CA_CERT\": \"<secret>\",\n \"HOST\": \"os-sample-your-project.aivencloud.com\",\n \"PASSWORD\": \"<secret>\",\n \"PORT\": \"14609\",\n \"USERNAME\": \"os-service-user\"\n}\nYou can connect to the OpenSearch instance using these credentials and the host information from the os-secret Secret.
PostgreSQL is an open source, relational database. It's ideal for organisations that need a well organised tabular datastore. On top of the strict table and columns formats, PostgreSQL also offers solutions for nested datasets with the native jsonb format and advanced set of extensions including PostGIS, a spatial database extender for location queries. Aiven for PostgreSQL is the perfect fit for your relational data.
With Aiven Kubernetes Operator, you can manage Aiven for PostgreSQL through the well defined Kubernetes API.
Note
Before going through this guide, make sure you have a Kubernetes cluster with the operator installed (see instructions for helm or kubectl), and a Kubernetes Secret with an Aiven authentication token.
"},{"location":"resources/postgresql.html#create-a-postgresql-instance","title":"Create a PostgreSQL instance","text":"1. Create a file named pg-sample.yaml with the following content:
apiVersion: aiven.io/v1alpha1\nkind: PostgreSQL\nmetadata:\n name: pg-sample\nspec:\n # gets the authentication token from the `aiven-token` Secret\n authSecretRef:\n name: aiven-token\n key: token\n\n # outputs the PostgreSQL connection on the `pg-connection` Secret\n connInfoSecretTarget:\n name: pg-connection\n\n # add your Project name here\n project: PROJECT_NAME\n\n # cloud provider and plan of your choice\n # you can check all of the possibilities here https://aiven.io/pricing\n cloudName: google-europe-west1\n plan: startup-4\n\n # general Aiven configuration\n maintenanceWindowDow: friday\n maintenanceWindowTime: 23:00:00\n\n # specific PostgreSQL configuration\n userConfig:\n pg_version: \"11\"\n2. Create the service by applying the configuration:
kubectl apply -f pg-sample.yaml\n3. Review the resource you created with the following command:
kubectl get postgresqls.aiven.io pg-sample\nThe output is similar to the following:
NAME PROJECT REGION PLAN STATE\npg-sample your-project google-europe-west1 startup-4 RUNNING\nThe resource can stay in the BUILDING state for a couple of minutes. Once the state changes to RUNNING, you are ready to access it.
For your convenience, the operator automatically stores the PostgreSQL connection information in a Secret created with the name specified on the connInfoSecretTarget field.
kubectl describe secret pg-connection\nThe output is similar to the following:
Name: pg-connection\nNamespace: default\nAnnotations: <none>\n\nType: Opaque\n\nData\n====\nDATABASE_URI: 107 bytes\nPGDATABASE: 9 bytes\nPGHOST: 38 bytes\nPGPASSWORD: 16 bytes\nPGPORT: 5 bytes\nPGSSLMODE: 7 bytes\nPGUSER: 8 bytes\nYou can use the jq to quickly decode the Secret:
kubectl get secret pg-connection -o json | jq '.data | map_values(@base64d)'\nThe output is similar to the following:
{\n \"DATABASE_URI\": \"postgres://avnadmin:<secret-password>@pg-sample-your-project.aivencloud.com:13039/defaultdb?sslmode=require\",\n \"PGDATABASE\": \"defaultdb\",\n \"PGHOST\": \"pg-sample-your-project.aivencloud.com\",\n \"PGPASSWORD\": \"<secret-password>\",\n \"PGPORT\": \"13039\",\n \"PGSSLMODE\": \"require\",\n \"PGUSER\": \"avnadmin\"\n}\nYou can verify your PostgreSQL connection from a Kubernetes workload by deploying a Pod that runs the psql command.
1. Create a file named pod-psql.yaml
apiVersion: v1\nkind: Pod\nmetadata:\n name: psql-test-connection\nspec:\n restartPolicy: Never\n containers:\n - image: postgres:11-alpine\n name: postgres\n command: [\"psql\", \"$(DATABASE_URI)\", \"-c\", \"SELECT version();\"]\n\n # the pg-connection Secret becomes environment variables\n envFrom:\n - secretRef:\n name: pg-connection\nIt runs once and stops, due to the restartPolicy: Never flag.
2. Inspect the log:
kubectl logs psql-test-connection\nThe output is similar to the following:
version\n---------------------------------------------------------------------------------------------\n PostgreSQL 11.12 on x86_64-pc-linux-gnu, compiled by gcc, a 68c5366192 p 6b9244f01a, 64-bit\n(1 row)\nYou have now connected to the PostgreSQL, and executed the SELECT version(); query.
The Database Kubernetes resource allows you to create a logical database within the PostgreSQL instance.
Create the pg-database-sample.yaml file with the following content:
apiVersion: aiven.io/v1alpha1\nkind: Database\nmetadata:\n name: pg-database-sample\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n # the name of the previously created PostgreSQL instance\n serviceName: pg-sample\n\n project: PROJECT_NAME\n lcCollate: en_US.UTF-8\n lcCtype: en_US.UTF-8\nYou can now connect to the pg-database-sample using the credentials stored in the pg-connection Secret.
Aiven uses the concept of service user that allows you to create users for different services. You can create one for the PostgreSQL instance.
1. Create a file named pg-service-user.yaml.
apiVersion: aiven.io/v1alpha1\nkind: ServiceUser\nmetadata:\n name: pg-service-user\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n connInfoSecretTarget:\n name: pg-service-user-connection\n\n project: PROJECT_NAME\n serviceName: pg-sample\n2. Apply the configuration with the following command.
kubectl apply -f pg-service-user.yaml\nThe ServiceUser resource generates a Secret with connection information, in this case named pg-service-user-connection:
kubectl get secret pg-service-user-connection -o json | jq '.data | map_values(@base64d)'\nThe output has the password and username:
{\n \"PASSWORD\": \"<secret-password>\",\n \"USERNAME\": \"pg-service-user\"\n}\nYou can now connect to the PostgreSQL instance using the credentials generated above, and the host information from the pg-connection Secret.
Connection pooling allows you to maintain very large numbers of connections to a database while minimizing the consumption of server resources. For more information, refer to the connection pooling article in Aiven Docs. Aiven for PostgreSQL uses PGBouncer for connection pooling.
You can create a connection pool with the ConnectionPool resource using the previously created Database and ServiceUser:
Create a new file named pg-connection-pool.yaml with the following content:
apiVersion: aiven.io/v1alpha1\nkind: ConnectionPool\nmetadata:\n name: pg-connection-pool\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n connInfoSecretTarget:\n name: pg-connection-pool-connection\n\n project: PROJECT_NAME\n serviceName: pg-sample\n databaseName: pg-database-sample\n username: pg-service-user\n poolSize: 10\n poolMode: transaction\nThe ConnectionPool generates a Secret with the connection info using the name from the connInfoSecretTarget.Name field:
kubectl get secret pg-connection-pool-connection -o json | jq '.data | map_values(@base64d)'\nThe output is similar to the following:
{\n \"DATABASE_URI\": \"postgres://pg-service-user:<secret-password>@pg-sample-you-project.aivencloud.com:13040/pg-connection-pool?sslmode=require\",\n \"PGDATABASE\": \"pg-database-sample\",\n \"PGHOST\": \"pg-sample-your-project.aivencloud.com\",\n \"PGPASSWORD\": \"<secret-password>\",\n \"PGPORT\": \"13040\",\n \"PGSSLMODE\": \"require\",\n \"PGUSER\": \"pg-service-user\"\n}\nRead-only replicas can be used to reduce the load on the primary service by making read-only queries against the replica service.
To create a read-only replica for a PostgreSQL service, you create a second PostgreSQL service and use serviceIntegrations to replicate data from your primary service.
The example that follows creates a primary service and a read-only replica.
1. Create a new file named pg-read-replica.yaml with the following:
apiVersion: aiven.io/v1alpha1\nkind: PostgreSQL\nmetadata:\n name: primary-pg-service\nspec:\n # gets the authentication token from the `aiven-token` Secret\n authSecretRef:\n name: aiven-token\n key: token\n\n # add your project's name here\n project: PROJECT_NAME\n\n # add the cloud provider and plan of your choice\n # you can see all of the options at https://aiven.io/pricing\n cloudName: google-europe-west1\n plan: startup-4\n\n # general Aiven configuration\n maintenanceWindowDow: friday\n maintenanceWindowTime: 23:00:00\n userConfig:\n pg_version: \"15\"\n\n---\napiVersion: aiven.io/v1alpha1\nkind: PostgreSQL\nmetadata:\n name: read-replica-pg\nspec:\n # gets the authentication token from the `aiven-token` Secret\n authSecretRef:\n name: aiven-token\n key: token\n\n # add your project's name here\n project: PROJECT_NAME\n\n # add the cloud provider and plan of your choice\n # you can see all of the options at https://aiven.io/pricing\n cloudName: google-europe-west1\n plan: startup-4\n\n # general Aiven configuration\n maintenanceWindowDow: saturday\n maintenanceWindowTime: 23:00:00\n userConfig:\n pg_version: \"15\"\n\n # use the read_replica integration and point it to your primary service\n serviceIntegrations:\n - integrationType: read_replica\n sourceServiceName: primary-pg-service\nNote
You can create the replica service in a different region or on a different cloud provider.
2. Apply the configuration with the following command:
kubectl apply -f pg-read-replica.yaml\nThe output is similar to the following:
postgresql.aiven.io/primary-pg-service created\npostgresql.aiven.io/read-replica-pg created\n3. Check the status of the primary service with the following command:
kubectl get postgresqls.aiven.io primary-pg-service\nThe output is similar to the following:
NAME PROJECT REGION PLAN STATE\nprimary-pg-service PROJECT_NAME google-europe-west1 startup-4 RUNNING\nThe resource can be in the BUILDING state for a few minutes. After the state of the primary service changes to RUNNING, the read-only replica is created. You can check the status of the replica using the same command with the name of the replica:
kubectl get postgresqls.aiven.io read-replica-pg\nVirtual Private Cloud (VPC) peering is a method of connecting separate AWS, Google Cloud or Microsoft Azure private networks to each other. It makes it possible for the virtual machines in the different VPCs to talk to each other directly without going through the public internet.
Within the Aiven Kubernetes Operator, you can create a ProjectVPC on Aiven's side to connect to your cloud provider.
Note
Before going through this guide, make sure you have a Kubernetes cluster with the operator installed (see instructions for helm or kubectl), and a Kubernetes Secret with an Aiven authentication token.
"},{"location":"resources/project-vpc.html#create-an-aiven-vpc","title":"Create an Aiven VPC","text":"1. Create a file named vpc-sample.yaml with the following content:
apiVersion: aiven.io/v1alpha1\nkind: ProjectVPC\nmetadata:\n name: vpc-sample\nspec:\n # gets the authentication token from the `aiven-token` Secret\n authSecretRef:\n name: aiven-token\n key: token\n\n project: PROJECT_NAME\n\n # creates a VPC to link an AWS account on the South Africa region\n cloudName: aws-af-south-1\n\n # the network range used by the VPC\n networkCidr: 192.168.0.0/24\n2. Create the Project by applying the configuration:
kubectl apply -f vpc-sample.yaml\n3. Review the resource you created with the following command:
kubectl get projects.aiven.io vpc-sample\nThe output is similar to the following:
NAME PROJECT CLOUD NETWORK CIDR\nvpc-sample PROJECT_NAME aws-af-south-1 192.168.0.0/24\nFollow the official VPC documentation to complete the VPC peering on your cloud of choice.
"},{"location":"resources/project.html","title":"Aiven Project","text":"Note
Before going through this guide, make sure you have a Kubernetes cluster with the operator installed (see instructions for helm or kubectl), and a Kubernetes Secret with an Aiven authentication token.
The Project CRD allows you to create Aiven Projects, where your resources can be located.
To create a fully working Aiven Project with the Aiven Operator you need a source Aiven Project already created with a working billing configuration, like a credit card.
Create a file named project-sample.yaml with the following content:
apiVersion: aiven.io/v1alpha1\nkind: Project\nmetadata:\n name: project-sample\nspec:\n # the source Project to copy the billing information from\n copyFromProject: SOURCE_PROJECT\n\n authSecretRef:\n name: aiven-token\n key: token\n\n connInfoSecretTarget:\n name: project-sample\nApply the resource with:
kubectl apply -f project-sample.yaml\nVerify the newly created Project:
kubectl get projects.aiven.io project-sample\nThe output is similar to the following:
NAME AGE\nproject-sample 22s\nAiven for Redis\u00ae* is a fully managed in-memory NoSQL database that you can deploy in the cloud of your choice to store and access data quickly and efficiently.
Note
Before going through this guide, make sure you have a Kubernetes cluster with the operator installed (see instructions for helm or kubectl), and a Kubernetes Secret with an Aiven authentication token.
"},{"location":"resources/redis.html#create-a-redis-instance","title":"Create a Redis instance","text":"1. Create a file named redis-sample.yaml, and add the following content:
apiVersion: aiven.io/v1alpha1\nkind: Redis\nmetadata:\n name: redis-sample\nspec:\n # gets the authentication token from the `aiven-token` Secret\n authSecretRef:\n name: aiven-token\n key: token\n\n # outputs the Redis connection on the `redis-secret` Secret\n connInfoSecretTarget:\n name: redis-secret\n\n # add your Project name here\n project: PROJECT_NAME\n\n # cloud provider and plan of your choice\n # you can check all of the possibilities here https://aiven.io/pricing\n cloudName: google-europe-west1\n plan: startup-4\n\n # general Aiven configuration\n maintenanceWindowDow: friday\n maintenanceWindowTime: 23:00:00\n\n # specific Redis configuration\n userConfig:\n redis_maxmemory_policy: \"allkeys-random\"\n2. Create the service by applying the configuration:
kubectl apply -f redis-sample.yaml\n3. Review the resource you created with this command:
kubectl describe redis.aiven.io redis-sample\nThe output is similar to the following:
...\nStatus:\n Conditions:\n Last Transition Time: 2023-01-19T14:48:59Z\n Message: Successfully created or updated the instance in Aiven\n Reason: Created\n Status: True\n Type: Initialized\n Last Transition Time: 2023-01-19T14:48:59Z\n Message: Successfully created or updated the instance in Aiven, status remains unknown\n Reason: Created\n Status: Unknown\n Type: Running\n State: REBUILDING\n...\nThe resource will be in the REBUILDING state for a few minutes. Once the state changes to RUNNING, you can access the resource.
For your convenience, the operator automatically stores the Redis connection information in a Secret created with the name specified on the connInfoSecretTarget field.
To view the details of the Secret, use the following command:
kubectl describe secret redis-secret\nThe output is similar to the following:
Name: redis-secret\nNamespace: default\nLabels: <none>\nAnnotations: <none>\n\nType: Opaque\n\nData\n====\nSSL: 8 bytes\nUSER: 7 bytes\nHOST: 60 bytes\nPASSWORD: 24 bytes\nPORT: 5 bytes\nYou can use the jq to quickly decode the Secret:
kubectl get secret redis-secret -o json | jq '.data | map_values(@base64d)'\nThe output is similar to the following:
{\n \"HOST\": \"redis-sample-your-project.aivencloud.com\",\n \"PASSWORD\": \"<secret-password>\",\n \"PORT\": \"14610\",\n \"SSL\": \"required\",\n \"USER\": \"default\"\n}\nService Integrations provide additional functionality and features by connecting different Aiven services together.
See our Getting Started with Service Integrations guide for more information.
Note
Before going through this guide, make sure you have a Kubernetes cluster with the operator installed (see instructions for helm or kubectl), and a Kubernetes Secret with an Aiven authentication token.
"},{"location":"resources/service-integrations.html#send-kafka-logs-to-a-kafka-topic","title":"Send Kafka logs to a Kafka Topic","text":"This integration allows you to send Kafka service logs to a specific Kafka Topic.
First, let's create a Kafka service and a topic.
1. Create a new file named kafka-sample-topic.yaml with the following content:
apiVersion: aiven.io/v1alpha1\nkind: Kafka\nmetadata:\n name: kafka-sample\nspec:\n # gets the authentication token from the `aiven-token` Secret\n authSecretRef:\n name: aiven-token\n key: token\n\n # outputs the Kafka connection on the `kafka-connection` Secret\n connInfoSecretTarget:\n name: kafka-auth\n\n # add your Project name here\n project: PROJECT_NAME\n\n # cloud provider and plan of your choice\n # you can check all of the possibilities here https://aiven.io/pricing\n cloudName: google-europe-west1\n plan: startup-2\n\n # general Aiven configuration\n maintenanceWindowDow: friday\n maintenanceWindowTime: 23:00:00\n\n # specific Kafka configuration\n userConfig:\n kafka_version: \"2.7\"\n\n---\napiVersion: aiven.io/v1alpha1\nkind: KafkaTopic\nmetadata:\n name: logs\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n project: PROJECT_NAME\n serviceName: kafka-sample\n\n # here we can specify how many partitions the topic should have\n partitions: 3\n # and the topic replication factor\n replication: 2\n\n # we also support various topic-specific configurations\n config:\n flush_ms: 100\n2. Create the resource on Kubernetes:
kubectl apply -f kafka-sample-topic.yaml\n3. Now, create a ServiceIntegration resource to send the Kafka logs to the created topic. In the same file, add the following YAML:
apiVersion: aiven.io/v1alpha1\nkind: ServiceIntegration\nmetadata:\n name: service-integration-kafka-logs\nspec:\n # gets the authentication token from the `aiven-token` Secret\n authSecretRef:\n name: aiven-token\n key: token\n\n project: PROJECT_NAME\n\n # indicates the type of the integration\n integrationType: kafka_logs\n\n # we will send the logs to the same kafka-sample instance\n # the source and destination are the same\n sourceServiceName: kafka-sample\n destinationServiceName: kafka-sample\n\n # the topic name we will send to\n kafkaLogs:\n kafka_topic: logs\n4. Reapply the resource on Kubernetes:
kubectl apply -f kafka-sample-topic.yaml\n5. Let's check the created service integration:
kubectl get serviceintegrations.aiven.io service-integration-kafka-logs\nThe output is similar to the following:
NAME PROJECT TYPE SOURCE SERVICE NAME DESTINATION SERVICE NAME SOURCE ENDPOINT ID DESTINATION ENDPOINT ID\nservice-integration-kafka-logs your-project kafka_logs kafka-sample kafka-sample\nYour Kafka service logs are now being streamed to the logs Kafka topic.
Aiven for Apache Kafka is an excellent option if you need to run Apache Kafka at scale. With Aiven Kubernetes Operator you can get up and running with a suitably sized Apache Kafka service in a few minutes.
Note
Before going through this guide, make sure you have a Kubernetes cluster with the operator installed (see instructions for helm or kubectl), and a Kubernetes Secret with an Aiven authentication token.
"},{"location":"resources/kafka/index.html#create-a-kafka-instance","title":"Create a Kafka instance","text":"1. Create a file named kafka-sample.yaml, and add the following content:
apiVersion: aiven.io/v1alpha1\nkind: Kafka\nmetadata:\n name: kafka-sample\nspec:\n # gets the authentication token from the `aiven-token` Secret\n authSecretRef:\n name: aiven-token\n key: token\n\n # outputs the Kafka connection on the `kafka-connection` Secret\n connInfoSecretTarget:\n name: kafka-auth\n\n # add your Project name here\n project: PROJECT_NAME\n\n # cloud provider and plan of your choice\n # you can check all of the possibilities here https://aiven.io/pricing\n cloudName: google-europe-west1\n plan: startup-2\n\n # general Aiven configuration\n maintenanceWindowDow: friday\n maintenanceWindowTime: 23:00:00\n\n # specific Kafka configuration\n userConfig:\n kafka_version: \"2.7\"\n2. Create the following resource on Kubernetes:
kubectl apply -f kafka-sample.yaml\n3. Inspect the service created using the command below.
kubectl get kafka.aiven.io kafka-sample\nThe output has the project name and state, similar to the following:
NAME PROJECT REGION PLAN STATE\nkafka-sample PROJECT_NAME google-europe-west1 startup-2 RUNNING\nAfter a couple of minutes, the STATE field is changed to RUNNING, and is ready to be used.
For your convenience, the operator automatically stores the Kafka connection information in a Secret created with the name specified on the connInfoSecretTarget field.
kubectl describe secret kafka-auth\nThe output is similar to the following:
Name: kafka-auth\nNamespace: default\nAnnotations: <none>\n\nType: Opaque\n\nData\n====\nCA_CERT: 1537 bytes\nHOST: 41 bytes\nPASSWORD: 16 bytes\nPORT: 5 bytes\nUSERNAME: 8 bytes\nACCESS_CERT: 1533 bytes\nACCESS_KEY: 2484 bytes\nYou can use the jq to quickly decode the Secret:
kubectl get secret kafka-auth -o json | jq '.data | map_values(@base64d)'\nThe output is similar to the following:
{\n \"CA_CERT\": \"<secret-ca-cert>\",\n \"ACCESS_CERT\": \"<secret-cert>\",\n \"ACCESS_KEY\": \"<secret-access-key>\",\n \"HOST\": \"kafka-sample-your-project.aivencloud.com\",\n \"PASSWORD\": \"<secret-password>\",\n \"PORT\": \"13041\",\n \"USERNAME\": \"avnadmin\"\n}\nYou can verify your access to the Kafka cluster from a Pod using the authentication data from the kafka-auth Secret. kcat is used for our examples below.
1. Create a file named kafka-test-connection.yaml, and add the following content:
apiVersion: v1\nkind: Pod\nmetadata:\n name: kafka-test-connection\nspec:\n restartPolicy: Never\n containers:\n - image: edenhill/kcat:1.7.0\n name: kcat\n\n # the command below will connect to the Kafka cluster\n # and output its metadata\n command:\n [\n \"kcat\",\n \"-b\",\n \"$(HOST):$(PORT)\",\n \"-X\",\n \"security.protocol=SSL\",\n \"-X\",\n \"ssl.key.location=/kafka-auth/ACCESS_KEY\",\n \"-X\",\n \"ssl.key.password=$(PASSWORD)\",\n \"-X\",\n \"ssl.certificate.location=/kafka-auth/ACCESS_CERT\",\n \"-X\",\n \"ssl.ca.location=/kafka-auth/CA_CERT\",\n \"-L\",\n ]\n\n # loading the data from the Secret as environment variables\n # useful to access the Kafka information, like hostname and port\n envFrom:\n - secretRef:\n name: kafka-auth\n\n volumeMounts:\n - name: kafka-auth\n mountPath: \"/kafka-auth\"\n\n # loading the data from the Secret as files in a volume\n # useful to access the Kafka certificates\n volumes:\n - name: kafka-auth\n secret:\n secretName: kafka-auth\n2. Apply the file.
kubectl apply -f kafka-test-connection.yaml\nOnce successfully applied, you have a log with the metadata information about the Kafka cluster.
kubectl logs kafka-test-connection\nThe output is similar to the following:
Metadata for all topics (from broker -1: ssl://kafka-sample-your-project.aivencloud.com:13041/bootstrap):\n 3 brokers:\n broker 2 at 35.205.234.70:13041\n broker 3 at 34.77.127.70:13041 (controller)\n broker 1 at 34.78.146.156:13041\n 0 topics:\nKafkaTopic and KafkaACL","text":"To properly produce and consume content on Kafka, you need topics and ACLs. The operator supports both with the KafkaTopic and KafkaACL resources.
Below, here is how to create a Kafka topic named random-strings where random string messages will be sent.
1. Create a file named kafka-topic-random-strings.yaml with the content below:
apiVersion: aiven.io/v1alpha1\nkind: KafkaTopic\nmetadata:\n name: random-strings\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n project: PROJECT_NAME\n serviceName: kafka-sample\n\n # here we can specify how many partitions the topic should have\n partitions: 3\n # and the topic replication factor\n replication: 2\n\n # we also support various topic-specific configurations\n config:\n flush_ms: 100\n2. Create the resource on Kubernetes:
kubectl apply -f kafka-topic-random-strings.yaml\n3. Create a user and an ACL. To use the Kafka topic, create a new user with the ServiceUser resource (in order to avoid using the avnadmin superuser), and the KafkaACL to allow the user access to the topic.
In a file named kafka-acl-user-crab.yaml, add the following two resources:
apiVersion: aiven.io/v1alpha1\nkind: ServiceUser\nmetadata:\n # the name of our user \ud83e\udd80\n name: crab\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n # the Secret name we will store the users' connection information\n # looks exactly the same as the Secret generated when creating the Kafka cluster\n # we will use this Secret to produce and consume events later!\n connInfoSecretTarget:\n name: kafka-crab-connection\n\n # the Aiven project the user is related to\n project: PROJECT_NAME\n\n # the name of our Kafka Service\n serviceName: kafka-sample\n\n---\napiVersion: aiven.io/v1alpha1\nkind: KafkaACL\nmetadata:\n name: crab\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n project: PROJECT_NAME\n serviceName: kafka-sample\n\n # the username from the ServiceUser above\n username: crab\n\n # the ACL allows to produce and consume on the topic\n permission: readwrite\n\n # specify the topic we created before\n topic: random-strings\nTo create the crab user and its permissions, execute the following command:
kubectl apply -f kafka-acl-user-crab.yaml\nUsing the previously created KafkaTopic, ServiceUser, KafkaACL, you can produce and consume events.
You can use kcat to produce a message into Kafka, and the -t random-strings argument to select the desired topic, and use the content of the /etc/issue file as the message's body.
1. Create a kafka-crab-produce.yaml file with the content below:
apiVersion: v1\nkind: Pod\nmetadata:\n name: kafka-crab-produce\nspec:\n restartPolicy: Never\n containers:\n - image: edenhill/kcat:1.7.0\n name: kcat\n\n # the command below will produce a message with the /etc/issue file content\n command:\n [\n \"kcat\",\n \"-b\",\n \"$(HOST):$(PORT)\",\n \"-X\",\n \"security.protocol=SSL\",\n \"-X\",\n \"ssl.key.location=/crab-auth/ACCESS_KEY\",\n \"-X\",\n \"ssl.key.password=$(PASSWORD)\",\n \"-X\",\n \"ssl.certificate.location=/crab-auth/ACCESS_CERT\",\n \"-X\",\n \"ssl.ca.location=/crab-auth/CA_CERT\",\n \"-P\",\n \"-t\",\n \"random-strings\",\n \"/etc/issue\",\n ]\n\n # loading the crab user data from the Secret as environment variables\n # useful to access the Kafka information, like hostname and port\n envFrom:\n - secretRef:\n name: kafka-crab-connection\n\n volumeMounts:\n - name: crab-auth\n mountPath: \"/crab-auth\"\n\n # loading the crab user information from the Secret as files in a volume\n # useful to access the Kafka certificates\n volumes:\n - name: crab-auth\n secret:\n secretName: kafka-crab-connection\n2. Create the Pod with the following content:
kubectl apply -f kafka-crab-produce.yaml\nNow your event is stored in Kafka.
To consume a message, you can use a graphical interface called Kowl. It allows you to explore information about our Kafka cluster, such as brokers, topics, or consumer groups.
1. Create a Kubernetes Pod and service to deploy and access Kowl. Create a file named kafka-crab-consume.yaml with the content below:
apiVersion: v1\nkind: Pod\nmetadata:\n name: kafka-crab-consume\n labels:\n app: kafka-crab-consume\nspec:\n containers:\n - image: quay.io/cloudhut/kowl:v1.4.0\n name: kowl\n\n # kowl configuration values\n env:\n - name: KAFKA_TLS_ENABLED\n value: \"true\"\n\n - name: KAFKA_BROKERS\n value: $(HOST):$(PORT)\n - name: KAFKA_TLS_PASSPHRASE\n value: $(PASSWORD)\n\n - name: KAFKA_TLS_CAFILEPATH\n value: /crab-auth/CA_CERT\n - name: KAFKA_TLS_CERTFILEPATH\n value: /crab-auth/ACCESS_CERT\n - name: KAFKA_TLS_KEYFILEPATH\n value: /crab-auth/ACCESS_KEY\n\n # inject all connection information as environment variables\n envFrom:\n - secretRef:\n name: kafka-crab-connection\n\n volumeMounts:\n - name: crab-auth\n mountPath: /crab-auth\n\n # loading the crab user information from the Secret as files in a volume\n # useful to access the Kafka certificates\n volumes:\n - name: crab-auth\n secret:\n secretName: kafka-crab-connection\n\n---\n# we will be using a simple service to access Kowl on port 8080\napiVersion: v1\nkind: Service\nmetadata:\n name: kafka-crab-consume\nspec:\n selector:\n app: kafka-crab-consume\n ports:\n - port: 8080\n targetPort: 8080\n2. Create the resources with:
kubectl apply -f kafka-crab-consume.yaml\n3. In another terminal create a port-forward tunnel to your Pod:
kubectl port-forward kafka-crab-consume 8080:8080\n4. In the browser of your choice, access the http://localhost:8080 address. You now see a page with the random-strings topic listed:
5. Click the topic name to see the message.
You have now consumed the message.
"},{"location":"resources/kafka/connect.html","title":"Kafka Connect","text":"Aiven for Apache Kafka Connect is a framework and a runtime for integrating Kafka with other systems. Kafka connectors can either be a source (for pulling data from other systems into Kafka) or sink (for pushing data into other systems from Kafka).
This section involves a few different Kubernetes CRDs:
KafkaService service with a KafkaTopicKafkaConnect serviceServiceIntegration to integrate the Kafka and KafkaConnect servicesPostgreSQL used as a sink to receive messages from KafkaKafkaConnector to finally connect the Kafka with the PostgreSQLCreate a file named kafka-sample-connect.yaml with the following content:
apiVersion: aiven.io/v1alpha1\nkind: Kafka\nmetadata:\n name: kafka-sample-connect\nspec:\n # gets the authentication token from the `aiven-token` Secret\n authSecretRef:\n name: aiven-token\n key: token\n\n # outputs the Kafka connection on the `kafka-connection` Secret\n connInfoSecretTarget:\n name: kafka-auth\n\n # add your Project name here\n project: PROJECT_NAME\n\n # cloud provider and plan of your choice\n # you can check all of the possibilities here https://aiven.io/pricing\n cloudName: google-europe-west1\n plan: business-4\n\n # general Aiven configuration\n maintenanceWindowDow: friday\n maintenanceWindowTime: 23:00:00\n\n # specific Kafka configuration\n userConfig:\n kafka_version: \"2.7\"\n kafka_connect: true\n\n---\napiVersion: aiven.io/v1alpha1\nkind: KafkaTopic\nmetadata:\n name: kafka-topic-connect\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n project: PROJECT_NAME\n serviceName: kafka-sample-connect\n\n replication: 2\n partitions: 1\nNext, create a file named kafka-connect.yaml and add the following KafkaConnect resource:
apiVersion: aiven.io/v1alpha1\nkind: KafkaConnect\nmetadata:\n name: kafka-connect\nspec:\n # gets the authentication token from the `aiven-token` Secret\n authSecretRef:\n name: aiven-token\n key: token\n\n # add your Project name here\n project: PROJECT_NAME\n\n # cloud provider and plan of your choice\n # you can check all of the possibilities here https://aiven.io/pricing\n cloudName: google-europe-west1\n plan: startup-4\n\n # general Aiven configuration\n maintenanceWindowDow: friday\n maintenanceWindowTime: 23:00:00\nNow let's create a ServiceIntegration. It will use the fields sourceServiceName and destinationServiceName to integrate the previously created kafka-sample-connect and kafka-connect. Open a new file named service-integration-connect.yaml and add the content below:
apiVersion: aiven.io/v1alpha1\nkind: ServiceIntegration\nmetadata:\n name: service-integration-kafka-connect\nspec:\n # gets the authentication token from the `aiven-token` Secret\n authSecretRef:\n name: aiven-token\n key: token\n\n project: PROJECT_NAME\n\n # indicates the type of the integration\n integrationType: kafka_connect\n\n # we will send messages from the `kafka-sample-connect` to `kafka-connect`\n sourceServiceName: kafka-sample-connect\n destinationServiceName: kafka-connect\nLet's add an Aiven for PostgreSQL service. It will be the service used as a sink, receiving messages from the kafka-sample-connect cluster. Create a file named pg-sample-connect.yaml with the content below:
apiVersion: aiven.io/v1alpha1\nkind: PostgreSQL\nmetadata:\n name: pg-connect\nspec:\n # gets the authentication token from the `aiven-token` Secret\n authSecretRef:\n name: aiven-token\n key: token\n\n # outputs the PostgreSQL connection on the `pg-connection` Secret\n connInfoSecretTarget:\n name: pg-connection\n\n # add your Project name here\n project: PROJECT_NAME\n\n # cloud provider and plan of your choice\n # you can check all of the possibilities here https://aiven.io/pricing\n cloudName: google-europe-west1\n plan: startup-4\n\n # general Aiven configuration\n maintenanceWindowDow: friday\n maintenanceWindowTime: 23:00:00\nFinally, let's add the glue of everything: a KafkaConnector. As described in the specification, it will send receive messages from the kafka-sample-connect and send them to the pg-connect service. Check our official documentation for more connectors.
Create a file named kafka-connector-connect.yaml and with the content below:
apiVersion: aiven.io/v1alpha1\nkind: KafkaConnector\nmetadata:\n name: kafka-connector\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n project: PROJECT_NAME\n\n # the Kafka cluster name\n serviceName: kafka-sample-connect\n\n # the connector we will be using\n connectorClass: io.aiven.connect.jdbc.JdbcSinkConnector\n\n userConfig:\n auto.create: \"true\"\n\n # constructs the pg-connect connection information\n connection.url: 'jdbc:postgresql://{{ fromSecret \"pg-connection\" \"PGHOST\"}}:{{ fromSecret \"pg-connection\" \"PGPORT\" }}/{{ fromSecret \"pg-connection\" \"PGDATABASE\" }}'\n connection.user: '{{ fromSecret \"pg-connection\" \"PGUSER\" }}'\n connection.password: '{{ fromSecret \"pg-connection\" \"PGPASSWORD\" }}'\n\n # specify which topics it will watch\n topics: kafka-topic-connect\n\n key.converter: org.apache.kafka.connect.json.JsonConverter\n value.converter: org.apache.kafka.connect.json.JsonConverter\n value.converter.schemas.enable: \"true\"\nWith all the files created, apply the new Kubernetes resources:
kubectl apply \\\n -f kafka-sample-connect.yaml \\\n -f kafka-connect.yaml \\\n -f service-integration-connect.yaml \\\n -f pg-sample-connect.yaml \\\n -f kafka-connector-connect.yaml\nIt will take some time for all the services to be up and running. You can check their status with the following command:
kubectl get \\\n kafkas.aiven.io/kafka-sample-connect \\\n kafkaconnects.aiven.io/kafka-connect \\\n postgresqls.aiven.io/pg-connect \\\n kafkaconnectors.aiven.io/kafka-connector\nThe output is similar to the following:
NAME PROJECT REGION PLAN STATE\nkafka.aiven.io/kafka-sample-connect your-project google-europe-west1 business-4 RUNNING\n\nNAME STATE\nkafkaconnect.aiven.io/kafka-connect RUNNING\n\nNAME PROJECT REGION PLAN STATE\npostgresql.aiven.io/pg-connect your-project google-europe-west1 startup-4 RUNNING\n\nNAME SERVICE NAME PROJECT CONNECTOR CLASS STATE TASKS TOTAL TASKS RUNNING\nkafkaconnector.aiven.io/kafka-connector kafka-sample-connect your-project io.aiven.connect.jdbc.JdbcSinkConnector RUNNING 1 1\nThe deployment is finished when all services have the state RUNNING.
To test the connection integration, let's produce a Kafka message using kcat from within the Kubernetes cluster. We will deploy a Pod responsible for crafting a message and sending to the Kafka cluster, using the kafka-auth secret generate by the Kafka CRD.
Create a new file named kcat-connect.yaml and add the content below:
apiVersion: v1\nkind: Pod\nmetadata:\n name: kafka-message\nspec:\n containers:\n\n restartPolicy: Never\n - image: edenhill/kcat:1.7.0\n name: kcat\n\n command: ['/bin/sh']\n args: [\n '-c',\n 'echo {\\\"schema\\\":{\\\"type\\\":\\\"struct\\\",\\\"fields\\\":[{ \\\"field\\\": \\\"text\\\", \\\"type\\\": \\\"string\\\", \\\"optional\\\": false } ] }, \\\"payload\\\": { \\\"text\\\": \\\"Hello World\\\" } } > /tmp/msg;\n\n kcat\n -b $(HOST):$(PORT)\n -X security.protocol=SSL\n -X ssl.key.location=/kafka-auth/ACCESS_KEY\n -X ssl.key.password=$(PASSWORD)\n -X ssl.certificate.location=/kafka-auth/ACCESS_CERT\n -X ssl.ca.location=/kafka-auth/CA_CERT\n -P -t kafka-topic-connect /tmp/msg'\n ]\n\n envFrom:\n - secretRef:\n name: kafka-auth\n\n volumeMounts:\n - name: kafka-auth\n mountPath: \"/kafka-auth\"\n\n volumes:\n - name: kafka-auth\n secret:\n secretName: kafka-auth\nApply the file with:
kubectl apply -f kcat-connect.yaml\nThe Pod will execute the commands and finish. You can confirm its Completed state with:
kubectl get pod kafka-message\nThe output is similar to the following:
NAME READY STATUS RESTARTS AGE\nkafka-message 0/1 Completed 0 5m35s\nIf everything went smoothly, we should have our produced message in the PostgreSQL service. Let's check that out.
Create a file named psql-connect.yaml with the content below:
apiVersion: v1\nkind: Pod\nmetadata:\n name: psql-connect\nspec:\n restartPolicy: Never\n containers:\n - image: postgres:13\n name: postgres\n # \"kafka-topic-connect\" is the table automatically created by KafkaConnect\n command:\n [\n \"psql\",\n \"$(DATABASE_URI)\",\n \"-c\",\n 'SELECT * from \"kafka-topic-connect\";',\n ]\n\n envFrom:\n - secretRef:\n name: pg-connection\nApply the file with:
kubectl apply -f psql-connect.yaml\nAfter a couple of seconds, inspect its log with this command:
kubectl logs psql-connect\nThe output is similar to the following:
text\n-------------\n Hello World\n(1 row)\nTo clean up all the created resources, use the following command:
kubectl delete \\\n -f kafka-sample-connect.yaml \\\n -f kafka-connect.yaml \\\n -f service-integration-connect.yaml \\\n -f pg-sample-connect.yaml \\\n -f kafka-connector-connect.yaml \\\n -f kcat-connect.yaml \\\n -f psql-connect.yaml\nKafkaSchema","text":"Aiven develops and maintain Karapace, an open source implementation of Kafka REST and schema registry. Is available out of the box for our managed Kafka service.
The schema registry address and authentication is the same as the Kafka broker, the only different is the usage of the port 13044.
First, let's create an Aiven for Apache Kafka service.
1. Create a file named kafka-sample-schema.yaml and add the content below:
apiVersion: aiven.io/v1alpha1\nkind: Kafka\nmetadata:\n name: kafka-sample-schema\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n connInfoSecretTarget:\n name: kafka-auth\n\n project: PROJECT_NAME\n cloudName: google-europe-west1\n plan: startup-2\n maintenanceWindowDow: friday\n maintenanceWindowTime: 23:00:00\n\n userConfig:\n kafka_version: \"2.7\"\n\n # this flag enables the Schema registry\n schema_registry: true\n2. Apply the changes with the following command:
kubectl apply -f kafka-schema.yaml\nNow, let's create the schema itself.
1. Create a new file named kafka-sample-schema.yaml and add the YAML content below:
apiVersion: aiven.io/v1alpha1\nkind: KafkaSchema\nmetadata:\n name: kafka-schema\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n project: PROJECT_NAME\n serviceName: kafka-sample-schema\n\n # the name of the Schema\n subjectName: MySchema\n\n # the schema itself, in JSON format\n schema: |\n {\n \"type\": \"record\",\n \"name\": \"MySchema\",\n \"fields\": [\n {\n \"name\": \"field\",\n \"type\": \"string\"\n }\n ]\n }\n\n # sets the schema compatibility level\n compatibilityLevel: BACKWARD\n2. Create the schema with the command:
kubectl apply -f kafka-schema.yaml\n3. Review the resource you created with the following command:
kubectl get kafkaschemas.aiven.io kafka-schema\nThe output is similar to the following:
NAME SERVICE NAME PROJECT SUBJECT COMPATIBILITY LEVEL VERSION\nkafka-schema kafka-sample PROJECT_NAME MySchema BACKWARD 1\nNow you can follow the instructions to use a schema registry in Java on how to use the schema created.
"}]} \ No newline at end of file diff --git a/search/search_index.json b/search/search_index.json index a3a42b25..097835ce 100644 --- a/search/search_index.json +++ b/search/search_index.json @@ -1 +1 @@ -{"config":{"lang":["en"],"separator":"[\\s\\-]+","pipeline":["stopWordFilter"]},"docs":[{"location":"index.html","title":"Welcome to Aiven Operator for Kubernetes","text":"Provision and manage Aiven services from your Kubernetes cluster.
"},{"location":"index.html#what-is-aiven","title":"What is Aiven?","text":"Aiven offers managed services for the best open source data technologies, on a cloud of your choice.
We offer multiple cloud options because we believe that everyone should have access to great data platforms wherever they host their applications. Our customers tell us they love it because they know that they aren\u2019t locked in to one particular cloud platform for all their data needs.
"},{"location":"index.html#contributing","title":"Contributing","text":"The contribution guide covers everything you need to know about how you can contribute to Aiven Operator for Kubernetes. The developer guide will help you onboard as a developer.
"},{"location":"authentication.html","title":"Authenticating","text":"To get authenticated and authorized, set up the communication between the Aiven Operator for Kubernetes and Aiven by using a token stored in a Kubernetes Secret. You can then refer to the Secret name on every custom resource in the authSecretRef field.
Important
If you don't have an Aiven account yet, sign up here for a free trial \ud83e\udd80
1. Generate an authentication token
Refer to our documentation article to generate your authentication token.
2. Create the Kubernetes Secret
The following command creates a Secret named aiven-token with a token field containing the authentication token:
kubectl create secret generic aiven-token --from-literal=token=\"TOKEN\"\nWhen managing your Aiven resources, we will be using the created Secret in the authSecretRef field. It will look like the example below:
apiVersion: aiven.io/v1alpha1\nkind: PostgreSQL\nmetadata:\n name: pg-sample\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n [ ... ]\nAlso, note that within Aiven, all resources are conceptually inside a Project. By default, a random project name is generated when you signup, but you can also create new projects.
The Project name is required in most of the resources. It will look like the example below:
apiVersion: aiven.io/v1alpha1\nkind: PostgreSQL\nmetadata:\n name: pg-sample\nspec:\n project: PROJECT_NAME\n [ ... ]\nhttp.StatusBadRequest on ClickhouseGrant deletionClickhouseGrant invalid remote and local privileges comparisonClickhouseGrant: doesn't escape role name to grantClickhouseUser: password was reset due to an incorrect processing cycleClickhouseRole deletion error (missing database)ClickhouseGrant deletion errors (missing database, service, role)REBALANCING stateClickhouseGrantKafkaConnect field userConfig.secret_providers, type array: Configure external secret providers in order to reference external secrets in connector configurationKafka field userConfig.kafka_connect_secret_providers, type array: Configure external secret providers in order to reference external secrets in connector configurationKafka field userConfig.letsencrypt_sasl_privatelink, type boolean: Use Letsencrypt CA for Kafka SASL via PrivatelinkServiceIntegration field datadog.mirrormaker_custom_metrics, type array: List of custom metricsServiceIntegration field kafkaMirrormaker.kafka_mirrormaker.consumer_auto_offset_reset, type string: Set where consumer starts to consume dataServiceIntegration field kafkaMirrormaker.kafka_mirrormaker.consumer_max_poll_records, type integer: Set consumer max.poll.records. The default is 500PostgreSQL field userConfig.pgaudit: deprecatedServiceIntegrationEndpoint field externalPostgresql.ssl_mode: enum [allow, disable, prefer, require, verify-ca, verify-full] \u2192 [require, verify-ca, verify-full]ServiceIntegrationEndpointServiceIntegration flink_external_postgresql typeServiceIntegration field datadog.datadog_pgbouncer_enabled, type boolean: Enable Datadog PgBouncer Metric TrackingServiceIntegration deletion when instance has no id setdisk_space field validationproject, serviceName fields validationConnectionPool doesn't check service user preconditionCA_CERT secret key for Grafana, OpenSearch, Redis, and Clickhouse. Can't be used with these service types ddog-gov.com, us3.datadoghq.com, us5.datadoghq.com]`ServiceIntegrationEndpoint field externalKafka.ssl_endpoint_identification_algorithm: enum [, https] \u2192 [https]ClickhouseUser webhook. Doesn't do any validation or mutationKafka field userConfig.kafka_version: enum [3.4, 3.5, 3.6] \u2192 [3.4, 3.5, 3.6, 3.7]ServiceIntegrationEndpoint field datadog.site: enum [datadoghq.com, datadoghq.eu, ddog-gov.com, us3.datadoghq.com, us5.datadoghq.com] \u2192 `[ap1.datadoghq.com, datadoghq.com, datadoghq.eu,ClickhouseRoleKafkaACLKafkaSchemaRegistryACL kindClickhouseDatabase kindDatabase kind validations and default values with CRD validation rulesREDIS_CA_CERT, MYSQL_CA_CERT, etc.KafkaTopic field config.local_retention_bytes, type integer: local.retention.bytes valueKafkaTopic field config.local_retention_ms, type integer: local.retention.ms valueKafkaTopic field config.remote_storage_enable, type boolean: remote_storage_enableCassandra field userConfig.cassandra_version: pattern ^[0-9]+(\\.[0-9]+)?$Cassandra field userConfig.project_to_fork_from: pattern ^[a-z][-a-z0-9]{0,63}$|^$Cassandra field userConfig.service_to_fork_from: pattern ^[a-z][-a-z0-9]{0,63}$|^$Cassandra field userConfig.service_to_join_with: pattern ^[a-z][-a-z0-9]{0,63}$Clickhouse field userConfig.project_to_fork_from: pattern ^[a-z][-a-z0-9]{0,63}$|^$Clickhouse field userConfig.service_to_fork_from: pattern ^[a-z][-a-z0-9]{0,63}$|^$Grafana field userConfig.project_to_fork_from: pattern ^[a-z][-a-z0-9]{0,63}$|^$Grafana field userConfig.service_to_fork_from: pattern ^[a-z][-a-z0-9]{0,63}$|^$Kafka field userConfig.kafka.sasl_oauthbearer_expected_audience: pattern ^[^\\r\\n]*$Kafka field userConfig.kafka.sasl_oauthbearer_expected_issuer: pattern ^[^\\r\\n]*$Kafka field userConfig.kafka.sasl_oauthbearer_sub_claim_name: pattern ^[^\\r\\n]*$MySQL field userConfig.mysql.default_time_zone: pattern ^([-+][\\d:]*|[\\w/]*)$MySQL field userConfig.project_to_fork_from: pattern ^[a-z][-a-z0-9]{0,63}$|^$MySQL field userConfig.service_to_fork_from: pattern ^[a-z][-a-z0-9]{0,63}$|^$OpenSearch field userConfig.openid.client_id: pattern ^[^\\r\\n]*$OpenSearch field userConfig.openid.client_secret: pattern ^[^\\r\\n]*$OpenSearch field userConfig.openid.header: pattern ^[^\\r\\n]*$OpenSearch field userConfig.openid.jwt_header: pattern ^[^\\r\\n]*$OpenSearch field userConfig.openid.jwt_url_parameter: pattern ^[^\\r\\n]*$OpenSearch field userConfig.openid.roles_key: pattern ^[^\\r\\n]*$OpenSearch field userConfig.openid.scope: pattern ^[^\\r\\n]*$OpenSearch field userConfig.openid.subject_key: pattern ^[^\\r\\n]*$OpenSearch field userConfig.project_to_fork_from: pattern ^[a-z][-a-z0-9]{0,63}$|^$OpenSearch field userConfig.saml.idp_entity_id: pattern ^[^\\r\\n]*$OpenSearch field userConfig.saml.roles_key: pattern ^[^\\r\\n]*$OpenSearch field userConfig.saml.sp_entity_id: pattern ^[^\\r\\n]*$OpenSearch field userConfig.saml.subject_key: pattern ^[^\\r\\n]*$OpenSearch field userConfig.service_to_fork_from: pattern ^[a-z][-a-z0-9]{0,63}$|^$PostgreSQL field userConfig.pg.timezone: pattern ^[\\w/]*$PostgreSQL field userConfig.pg_service_to_fork_from: pattern ^[a-z][-a-z0-9]{0,63}$|^$PostgreSQL field userConfig.project_to_fork_from: pattern ^[a-z][-a-z0-9]{0,63}$|^$PostgreSQL field userConfig.service_to_fork_from: pattern ^[a-z][-a-z0-9]{0,63}$|^$Redis field userConfig.project_to_fork_from: pattern ^[a-z][-a-z0-9]{0,63}$|^$Redis field userConfig.service_to_fork_from: pattern ^[a-z][-a-z0-9]{0,63}$|^$OpenSearch field userConfig.opensearch.plugins_alerting_filter_by_backend_roles, type boolean: Enable or disable filtering of alerting by backend roles. Requires Security pluginRedis field userConfig.redis_notify_keyspace_events: pattern ^[KEg\\$lshzxeA]*$ \u2192 ^[KEg\\$lshzxentdmA]*$PostgreSQL field userConfig.pgaudit, type object: System-wide settings for the pgaudit extensionServiceIntegration field datadog.opensearch.cluster_stats_enabled, type boolean: Enable Datadog Opensearch Cluster MonitoringOpenSearch field userConfig.opensearch.enable_security_audit, type boolean: Enable/Disable security auditKafka field userConfig.kafka_rest_config.name_strategy, type string: Name strategy to use when selecting subject for storing schemasRedis field userConfig.redis_version, type string: Redis major versionGrafana field userConfig.auth_github.auto_login, type boolean: Allow users to bypass the login screen and automatically log inGrafana field userConfig.auth_github.skip_org_role_sync, type boolean: Stop automatically syncing user rolesClickhouse field userConfig.additional_backup_regions: deprecatedGrafana field userConfig.additional_backup_regions: deprecatedKafkaConnect field userConfig.additional_backup_regions: deprecatedKafka field userConfig.additional_backup_regions: deprecatedOpenSearch field userConfig.additional_backup_regions: deprecatedRedis field userConfig.additional_backup_regions: deprecatedCassandra field userConfig.cassandra_version: enum [3, 4, 4.1] \u2192 [4, 4.1]Kafka field userConfig.kafka_version: enum [3.1, 3.3, 3.4, 3.5, 3.6] \u2192 [3.4, 3.5, 3.6]PostgreSQL field userConfig.pg_version: enum [11, 12, 13, 14, 15, 16] \u2192 [12, 13, 14, 15, 16]Cassandra field technicalEmails, type array: Defines the email addresses that will receive alerts about upcoming maintenance updates or warnings about service instabilityClickhouse field technicalEmails, type array: Defines the email addresses that will receive alerts about upcoming maintenance updates or warnings about service instabilityGrafana field technicalEmails, type array: Defines the email addresses that will receive alerts about upcoming maintenance updates or warnings about service instabilityKafkaConnect field technicalEmails, type array: Defines the email addresses that will receive alerts about upcoming maintenance updates or warnings about service instabilityKafka field technicalEmails, type array: Defines the email addresses that will receive alerts about upcoming maintenance updates or warnings about service instabilityMySQL field technicalEmails, type array: Defines the email addresses that will receive alerts about upcoming maintenance updates or warnings about service instabilityOpenSearch field technicalEmails, type array: Defines the email addresses that will receive alerts about upcoming maintenance updates or warnings about service instabilityPostgreSQL field technicalEmails, type array: Defines the email addresses that will receive alerts about upcoming maintenance updates or warnings about service instabilityRedis field technicalEmails, type array: Defines the email addresses that will receive alerts about upcoming maintenance updates or warnings about service instabilityCassandra field connInfoSecretTargetDisabled, type boolean: When true, the secret containing connection information will not be created, defaults to falseClickhouse field connInfoSecretTargetDisabled, type boolean: When true, the secret containing connection information will not be created, defaults to falseClickhouseUser field connInfoSecretTargetDisabled, type boolean: When true, the secret containing connection information will not be created, defaults to falseConnectionPool field connInfoSecretTargetDisabled, type boolean: When true, the secret containing connection information will not be created, defaults to falseGrafana field connInfoSecretTargetDisabled, type boolean: When true, the secret containing connection information will not be created, defaults to falseKafka field connInfoSecretTargetDisabled, type boolean: When true, the secret containing connection information will not be created, defaults to falseMySQL field connInfoSecretTargetDisabled, type boolean: When true, the secret containing connection information will not be created, defaults to falseOpenSearch field connInfoSecretTargetDisabled, type boolean: When true, the secret containing connection information will not be created, defaults to falsePostgreSQL field connInfoSecretTargetDisabled, type boolean: When true, the secret containing connection information will not be created, defaults to falseProject field connInfoSecretTargetDisabled, type boolean: When true, the secret containing connection information will not be created, defaults to falseRedis field connInfoSecretTargetDisabled, type boolean: When true, the secret containing connection information will not be created, defaults to falseServiceUser field connInfoSecretTargetDisabled, type boolean: When true, the secret containing connection information will not be created, defaults to falseKAFKA_SCHEMA_REGISTRY_URI and KAFKA_REST_URI to Kafka secretCONNECTIONPOOL_NAME in ConnectionPool secretCONNECTIONPOOL_PORT exposes service port instead of pool portSERVICEUSER_PORT when sasl is the only authentication methodPostgreSQL field userConfig.pg_qualstats.enabled: deprecatedPostgreSQL field userConfig.pg_qualstats.min_err_estimate_num: deprecatedPostgreSQL field userConfig.pg_qualstats.min_err_estimate_ratio: deprecatedPostgreSQL field userConfig.pg_qualstats.track_constants: deprecatedPostgreSQL field userConfig.pg_qualstats.track_pg_catalog: deprecatedPreconditions, CreateOrUpdate, Delete. Thanks to @ataraxKafka field userConfig.kafka.transaction_partition_verification_enable, type boolean: Enable verification that checks that the partition has been added to the transaction before writing transactional records to the partitionCassandra field userConfig.service_log, type boolean: Store logs for the service so that they are available in the HTTP API and consoleClickhouse field userConfig.service_log, type boolean: Store logs for the service so that they are available in the HTTP API and consoleGrafana field userConfig.service_log, type boolean: Store logs for the service so that they are available in the HTTP API and consoleKafkaConnect field userConfig.service_log, type boolean: Store logs for the service so that they are available in the HTTP API and consoleKafka field userConfig.kafka_rest_config.name_strategy_validation, type boolean: If true, validate that given schema is registered under expected subject name by the used name strategy when producing messagesKafka field userConfig.service_log, type boolean: Store logs for the service so that they are available in the HTTP API and consoleMySQL field userConfig.service_log, type boolean: Store logs for the service so that they are available in the HTTP API and consoleOpenSearch field userConfig.service_log, type boolean: Store logs for the service so that they are available in the HTTP API and consolePostgreSQL field userConfig.pg_qualstats, type object: System-wide settings for the pg_qualstats extensionPostgreSQL field userConfig.service_log, type boolean: Store logs for the service so that they are available in the HTTP API and consoleRedis field userConfig.service_log, type boolean: Store logs for the service so that they are available in the HTTP API and consoleServiceIntegration: do not send empty user config to the APIstring type fields to the documentationClickhouse field userConfig.private_access.clickhouse_mysql, type boolean: Allow clients to connect to clickhouse_mysql with a DNS name that always resolves to the service's private IP addressesClickhouse field userConfig.privatelink_access.clickhouse_mysql, type boolean: Enable clickhouse_mysqlClickhouse field userConfig.public_access.clickhouse_mysql, type boolean: Allow clients to connect to clickhouse_mysql from the public internet for service nodes that are in a project VPC or another type of private networkGrafana field userConfig.unified_alerting_enabled, type boolean: Enable or disable Grafana unified alerting functionalityKafka field userConfig.aiven_kafka_topic_messages, type boolean: Allow access to read Kafka topic messages in the Aiven Console and REST APIKafka field userConfig.kafka.sasl_oauthbearer_expected_audience, type string: The (optional) comma-delimited setting for the broker to use to verify that the JWT was issued for one of the expected audiencesKafka field userConfig.kafka.sasl_oauthbearer_expected_issuer, type string: Optional setting for the broker to use to verify that the JWT was created by the expected issuerKafka field userConfig.kafka.sasl_oauthbearer_jwks_endpoint_url, type string: OIDC JWKS endpoint URL. By setting this the SASL SSL OAuth2/OIDC authentication is enabledKafka field userConfig.kafka.sasl_oauthbearer_sub_claim_name, type string: Name of the scope from which to extract the subject claim from the JWT. Defaults to subKafka field userConfig.kafka_version: enum [3.1, 3.3, 3.4, 3.5] \u2192 [3.1, 3.3, 3.4, 3.5, 3.6]Kafka field userConfig.tiered_storage.local_cache.size: deprecatedOpenSearch field userConfig.opensearch.indices_memory_max_index_buffer_size, type integer: Absolute value. Default is unbound. Doesn't work without indices.memory.index_buffer_sizeOpenSearch field userConfig.opensearch.indices_memory_min_index_buffer_size, type integer: Absolute value. Default is 48mb. Doesn't work without indices.memory.index_buffer_sizeOpenSearch field userConfig.opensearch.auth_failure_listeners.internal_authentication_backend_limiting.authentication_backend: enum [internal]OpenSearch field userConfig.opensearch.auth_failure_listeners.internal_authentication_backend_limiting.type: enum [username]OpenSearch field userConfig.opensearch.auth_failure_listeners.ip_rate_limiting.type: enum [ip]OpenSearch field userConfig.opensearch.search_max_buckets: maximum 65536 \u2192 1000000ServiceIntegration field kafkaMirrormaker.kafka_mirrormaker.producer_max_request_size: maximum 67108864 \u2192 268435456projectVpcId and projectVPCRef mutablenil user config conversionCassandra kind option additional_backup_regionsGrafana kind option auto_loginKafka kind properties log_local_retention_bytes, log_local_retention_msKafka kind option remote_log_storage_system_enableOpenSearch kind option auth_failure_listenersOpenSearch kind Index State Management optionsKafkaKafka version 3.5Kafka spec property scheduled_rebalance_max_delay_msKafka spec property remote_log_storage_system_enableKafkaConnect spec property scheduled_rebalance_max_delay_msOpenSearch spec property openidKAFKA_SCHEMA_REGISTRY_HOST and KAFKA_SCHEMA_REGISTRY_PORT for KafkaKAFKA_CONNECT_HOST, KAFKA_CONNECT_PORT, KAFKA_REST_HOST and KAFKA_REST_PORT for Kafka. Thanks to @Dariuschunclean_leader_election_enable from KafkaTopic kind configKAFKA_SASL_PORT for Kafka kind if SASL authentication method is enabledredis options to datadog ServiceIntegrationCassandra version 3Kafka versions 3.1 and 3.4kafka_rest_config.producer_max_request_size optionkafka_mirrormaker.producer_compression_type optionclusterRole.create option to Helm chart. Thanks to @ryaneorthOpenSearch.spec.userConfig.idp_pemtrustedcas_content option. Specifies the PEM-encoded root certificate authority (CA) content for the SAML identity provider (IdP) server verification.ServiceIntegration kind SourceProjectName and DestinationProjectName fieldsServiceIntegration fields MaxLength validationServiceIntegration validation: multiple user configs cannot be setServiceIntegration, should not require destinationServiceName or sourceEndpointID fieldServiceIntegration, add missing external_aws_cloudwatch_metrics type config serializationServiceIntegration integration type listannotations and labels fields to connInfoSecretTargetOpenSearch.spec.userConfig.opensearch.search_max_buckets maximum to 65536plan as a required fieldminumim, maximum validations for number typeip_filter backward compatibilityclickhouseKafka.tables.data_format-property enum RawBLOB valueuserConfig.opensearch.email_sender_username validation patternlog_cleaner_min_cleanable_ratio minimum and maximum validation rules3.2, reached EOL10, reached EOLProjectVPC by ID to avoid conflictsProjectVPC deletion by exiting on DELETING statusClickhouseUser controllerClickhouseUser.spec.project and ClickhouseUser.spec.serviceName as immutablesignalfxAuthSecretRef fields marked as requireddatadog, kafka_connect, kafka_logs, metricsclickhouse_postgresql, clickhouse_kafka, clickhouse_kafka, logs, external_aws_cloudwatch_metricsKafkaTopic.Spec.topicName field. Unlike the metadata.name, supports additional characters and has a longer length. KafkaTopic.Spec.topicName replaces metadata.name in future releases and will be marked as required.false value for termination_protection propertymin_cleanable_dirty_ratio. Thanks to @TV2rdImportant: This release brings breaking changes to the userConfig property. After new charts are installed, update your existing instances manually using the kubectl edit command according to the API reference.
Note: It is now recommended to disable webhooks for Kubernetes version 1.25 and higher, as native CRD validation rules are used.
ip_filter field is now of object typeserviceIntegrations on service types. Only the read_replica type is available.min_cleanable_dirty_ratio config field supportspec.disk_space propertylinux/amd64 build. Thanks to @christoffer-eidenever from choices of maintenance dowdevelopment flag to configure logger's behaviormake generate-user-configs)genericServiceHandler to generalize service managementKafkaACL deletionProjectVPCRef property to Kafka, OpenSearch, Clickhouse and Redis kinds to get ProjectVPC ID when resource is readyProjectVPC deletion, deletes by ID first if possible, then tries by nameclient.Object storage update data lossfeatures:
improvements:
fixes:
improvements:
fixes:
features:
Use the following checks to help you troubleshoot the Aiven Operator for Kubernetes.
"},{"location":"troubleshooting.html#check-the-pods","title":"Check the Pods","text":"Verify that all the operator Pods are READY, and the STATUS is Running.
kubectl get pod -n aiven-operator-system\nThe output is similar to the following:
NAME READY STATUS RESTARTS AGE\naiven-operator-controller-manager-576d944499-ggttj 1/1 Running 0 12m\nVerify that the cert-manager Pods are also running.
kubectl get pod -n cert-manager\nThe output has the status:
NAME READY STATUS RESTARTS AGE\ncert-manager-7dd5854bb4-85cpv 1/1 Running 0 76s\ncert-manager-cainjector-64c949654c-n2z8l 1/1 Running 0 77s\ncert-manager-webhook-6bdffc7c9d-47w6z 1/1 Running 0 76s\nUse the following command to visualize all the logs from the operator.
kubectl logs -n aiven-operator-system -l control-plane=controller-manager\nkubectl get pod -n aiven-operator-system -l control-plane=controller-manager -o jsonpath=\"{.items[0].spec.containers[0].image}\"\nWe're always working to resolve problems that pop up in Aiven products. If your problem is listed below, we know about it and are working to fix it. If your problem isn't listed below, report it as an issue.
"},{"location":"troubleshooting.html#cert-manager","title":"cert-manager","text":""},{"location":"troubleshooting.html#issue","title":"Issue","text":"The following event appears on the operator Pod:
MountVolume.SetUp failed for volume \"cert\" : secret \"webhook-server-cert\" not found\nYou cannot run the operator.
"},{"location":"troubleshooting.html#solution","title":"Solution","text":"Make sure that cert-manager is up and running.
kubectl get pod -n cert-manager\nThe output shows the status of each cert-manager:
NAME READY STATUS RESTARTS AGE\ncert-manager-7dd5854bb4-85cpv 1/1 Running 0 76s\ncert-manager-cainjector-64c949654c-n2z8l 1/1 Running 0 77s\ncert-manager-webhook-6bdffc7c9d-47w6z 1/1 Running 0 76s\nAutogenerated from CRD files.
"},{"location":"api-reference/cassandra.html","title":"Cassandra","text":""},{"location":"api-reference/cassandra.html#usage-example","title":"Usage example","text":"ExampleapiVersion: aiven.io/v1alpha1\nkind: Cassandra\nmetadata:\n name: my-cassandra\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n connInfoSecretTarget:\n name: cassandra-secret\n prefix: MY_SECRET_PREFIX_\n annotations:\n foo: bar\n labels:\n baz: egg\n\n project: aiven-project-name\n cloudName: google-europe-west1\n plan: startup-4\n\n maintenanceWindowDow: sunday\n maintenanceWindowTime: 11:00:00\n\n userConfig:\n migrate_sstableloader: true\n public_access:\n prometheus: true\n ip_filter:\n - network: 0.0.0.0\n description: whatever\n - network: 10.20.0.0/16\nInfo
To create this resource, a Secret containing Aiven token must be created first.
Apply the resource with:
kubectl apply -f example.yaml\nVerify the newly created Cassandra:
kubectl get cassandras my-cassandra\nThe output is similar to the following:
Name Project Region Plan State \nmy-cassandra aiven-project-name google-europe-west1 startup-4 RUNNING \nTo view the details of the Secret, use the following command:
kubectl describe secret cassandra-secret\nYou can use the jq to quickly decode the Secret:
kubectl get secret cassandra-secret -o json | jq '.data | map_values(@base64d)'\nThe output is similar to the following:
{\n \"CASSANDRA_HOST\": \"<secret>\",\n \"CASSANDRA_PORT\": \"<secret>\",\n \"CASSANDRA_USER\": \"<secret>\",\n \"CASSANDRA_PASSWORD\": \"<secret>\",\n \"CASSANDRA_URI\": \"<secret>\",\n \"CASSANDRA_HOSTS\": \"<secret>\",\n \"CASSANDRA_CA_CERT\": \"<secret>\",\n}\nCassandra is the Schema for the cassandras API.
Exposes secret keys
CASSANDRA_HOST, CASSANDRA_PORT, CASSANDRA_USER, CASSANDRA_PASSWORD, CASSANDRA_URI, CASSANDRA_HOSTS, CASSANDRA_CA_CERT.
Required
apiVersion (string). Value aiven.io/v1alpha1.kind (string). Value Cassandra.metadata (object). Data that identifies the object, including a name string and optional namespace.spec (object). CassandraSpec defines the desired state of Cassandra. See below for nested schema.Appears on Cassandra.
CassandraSpec defines the desired state of Cassandra.
Required
plan (string, MaxLength: 128). Subscription plan.project (string, Immutable, Pattern: ^[a-zA-Z0-9_-]+$, MaxLength: 63). Identifies the project this resource belongs to.Optional
authSecretRef (object). Authentication reference to Aiven token in a secret. See below for nested schema.cloudName (string, MaxLength: 256). Cloud the service runs in.connInfoSecretTarget (object). Secret configuration. See below for nested schema.connInfoSecretTargetDisabled (boolean, Immutable). When true, the secret containing connection information will not be created, defaults to false. This field cannot be changed after resource creation.disk_space (string, Pattern: (?i)^[1-9][0-9]*(GiB|G)?$). The disk space of the service, possible values depend on the service type, the cloud provider and the project. Reducing will result in the service re-balancing. The removal of this field does not change the value.maintenanceWindowDow (string, Enum: monday, tuesday, wednesday, thursday, friday, saturday, sunday). Day of week when maintenance operations should be performed. One monday, tuesday, wednesday, etc.maintenanceWindowTime (string, MaxLength: 8). Time of day when maintenance operations should be performed. UTC time in HH:mm:ss format.projectVPCRef (object). ProjectVPCRef reference to ProjectVPC resource to use its ID as ProjectVPCID automatically. See below for nested schema.projectVpcId (string, MaxLength: 36). Identifier of the VPC the service should be in, if any.serviceIntegrations (array of objects, Immutable, MaxItems: 1). Service integrations to specify when creating a service. Not applied after initial service creation. See below for nested schema.tags (object, AdditionalProperties: string). Tags are key-value pairs that allow you to categorize services.technicalEmails (array of objects, MaxItems: 10). Defines the email addresses that will receive alerts about upcoming maintenance updates or warnings about service instability. See below for nested schema.terminationProtection (boolean). Prevent service from being deleted. It is recommended to have this enabled for all services.userConfig (object). Cassandra specific user configuration options. See below for nested schema.Appears on spec.
Authentication reference to Aiven token in a secret.
Required
key (string, MinLength: 1).name (string, MinLength: 1).Appears on spec.
Secret configuration.
Required
name (string, Immutable). Name of the secret resource to be created. By default, it is equal to the resource name.Optional
annotations (object, AdditionalProperties: string). Annotations added to the secret.labels (object, AdditionalProperties: string). Labels added to the secret.prefix (string). Prefix for the secret's keys. Added \"as is\" without any transformations. By default, is equal to the kind name in uppercase + underscore, e.g. KAFKA_, REDIS_, etc.Appears on spec.
ProjectVPCRef reference to ProjectVPC resource to use its ID as ProjectVPCID automatically.
Required
name (string, MinLength: 1).Optional
namespace (string, MinLength: 1).Appears on spec.
Service integrations to specify when creating a service. Not applied after initial service creation.
Required
integrationType (string, Enum: read_replica).sourceServiceName (string, MinLength: 1, MaxLength: 64).Appears on spec.
Defines the email addresses that will receive alerts about upcoming maintenance updates or warnings about service instability.
Required
email (string). Email address.Appears on spec.
Cassandra specific user configuration options.
Optional
additional_backup_regions (array of strings, MaxItems: 1). Deprecated. Additional Cloud Regions for Backup Replication.backup_hour (integer, Minimum: 0, Maximum: 23). The hour of day (in UTC) when backup for the service is started. New backup is only started if previous backup has already completed.backup_minute (integer, Minimum: 0, Maximum: 59). The minute of an hour when backup for the service is started. New backup is only started if previous backup has already completed.cassandra (object). cassandra configuration values. See below for nested schema.cassandra_version (string, Enum: 4, 4.1, Pattern: ^[0-9]+(\\.[0-9]+)?$). Cassandra version.ip_filter (array of objects, MaxItems: 1024). Allow incoming connections from CIDR address block, e.g. 10.20.0.0/16. See below for nested schema.migrate_sstableloader (boolean). Sets the service into migration mode enabling the sstableloader utility to be used to upload Cassandra data files. Available only on service create.private_access (object). Allow access to selected service ports from private networks. See below for nested schema.project_to_fork_from (string, Immutable, Pattern: ^[a-z][-a-z0-9]{0,63}$|^$, MaxLength: 63). Name of another project to fork a service from. This has effect only when a new service is being created.public_access (object). Allow access to selected service ports from the public Internet. See below for nested schema.service_log (boolean). Store logs for the service so that they are available in the HTTP API and console.service_to_fork_from (string, Immutable, Pattern: ^[a-z][-a-z0-9]{0,63}$|^$, MaxLength: 64). Name of another service to fork from. This has effect only when a new service is being created.service_to_join_with (string, Pattern: ^[a-z][-a-z0-9]{0,63}$, MaxLength: 64). When bootstrapping, instead of creating a new Cassandra cluster try to join an existing one from another service. Can only be set on service creation.static_ips (boolean). Use static public IP addresses.Appears on spec.userConfig.
cassandra configuration values.
Optional
batch_size_fail_threshold_in_kb (integer, Minimum: 1, Maximum: 1000000). Fail any multiple-partition batch exceeding this value. 50kb (10x warn threshold) by default.batch_size_warn_threshold_in_kb (integer, Minimum: 1, Maximum: 1000000). Log a warning message on any multiple-partition batch size exceeding this value.5kb per batch by default.Caution should be taken on increasing the size of this thresholdas it can lead to node instability.datacenter (string, MaxLength: 128). Name of the datacenter to which nodes of this service belong. Can be set only when creating the service.Appears on spec.userConfig.
Allow incoming connections from CIDR address block, e.g. 10.20.0.0/16.
Required
network (string, MaxLength: 43). CIDR address block.Optional
description (string, MaxLength: 1024). Description for IP filter list entry.Appears on spec.userConfig.
Allow access to selected service ports from private networks.
Required
prometheus (boolean). Allow clients to connect to prometheus with a DNS name that always resolves to the service's private IP addresses. Only available in certain network locations.Appears on spec.userConfig.
Allow access to selected service ports from the public Internet.
Required
prometheus (boolean). Allow clients to connect to prometheus from the public internet for service nodes that are in a project VPC or another type of private network.apiVersion: aiven.io/v1alpha1\nkind: Clickhouse\nmetadata:\n name: my-clickhouse\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n connInfoSecretTarget:\n name: my-clickhouse\n annotations:\n foo: bar\n labels:\n baz: egg\n\n tags:\n env: test\n instance: foo\n\n userConfig:\n ip_filter:\n - network: 0.0.0.0/32\n description: bar\n - network: 10.20.0.0/16\n\n project: my-aiven-project\n cloudName: google-europe-west1\n plan: startup-16\n\n maintenanceWindowDow: friday\n maintenanceWindowTime: 23:00:00\nInfo
To create this resource, a Secret containing Aiven token must be created first.
Apply the resource with:
kubectl apply -f example.yaml\nVerify the newly created Clickhouse:
kubectl get clickhouses my-clickhouse\nThe output is similar to the following:
Name Project Region Plan State \nmy-clickhouse my-aiven-project google-europe-west1 startup-16 RUNNING \nTo view the details of the Secret, use the following command:
kubectl describe secret my-clickhouse\nYou can use the jq to quickly decode the Secret:
kubectl get secret my-clickhouse -o json | jq '.data | map_values(@base64d)'\nThe output is similar to the following:
{\n \"CLICKHOUSE_HOST\": \"<secret>\",\n \"CLICKHOUSE_PORT\": \"<secret>\",\n \"CLICKHOUSE_USER\": \"<secret>\",\n \"CLICKHOUSE_PASSWORD\": \"<secret>\",\n}\nClickhouse is the Schema for the clickhouses API.
Exposes secret keys
CLICKHOUSE_HOST, CLICKHOUSE_PORT, CLICKHOUSE_USER, CLICKHOUSE_PASSWORD.
Required
apiVersion (string). Value aiven.io/v1alpha1.kind (string). Value Clickhouse.metadata (object). Data that identifies the object, including a name string and optional namespace.spec (object). ClickhouseSpec defines the desired state of Clickhouse. See below for nested schema.Appears on Clickhouse.
ClickhouseSpec defines the desired state of Clickhouse.
Required
plan (string, MaxLength: 128). Subscription plan.project (string, Immutable, Pattern: ^[a-zA-Z0-9_-]+$, MaxLength: 63). Identifies the project this resource belongs to.Optional
authSecretRef (object). Authentication reference to Aiven token in a secret. See below for nested schema.cloudName (string, MaxLength: 256). Cloud the service runs in.connInfoSecretTarget (object). Secret configuration. See below for nested schema.connInfoSecretTargetDisabled (boolean, Immutable). When true, the secret containing connection information will not be created, defaults to false. This field cannot be changed after resource creation.disk_space (string, Pattern: (?i)^[1-9][0-9]*(GiB|G)?$). The disk space of the service, possible values depend on the service type, the cloud provider and the project. Reducing will result in the service re-balancing. The removal of this field does not change the value.maintenanceWindowDow (string, Enum: monday, tuesday, wednesday, thursday, friday, saturday, sunday). Day of week when maintenance operations should be performed. One monday, tuesday, wednesday, etc.maintenanceWindowTime (string, MaxLength: 8). Time of day when maintenance operations should be performed. UTC time in HH:mm:ss format.projectVPCRef (object). ProjectVPCRef reference to ProjectVPC resource to use its ID as ProjectVPCID automatically. See below for nested schema.projectVpcId (string, MaxLength: 36). Identifier of the VPC the service should be in, if any.serviceIntegrations (array of objects, Immutable, MaxItems: 1). Service integrations to specify when creating a service. Not applied after initial service creation. See below for nested schema.tags (object, AdditionalProperties: string). Tags are key-value pairs that allow you to categorize services.technicalEmails (array of objects, MaxItems: 10). Defines the email addresses that will receive alerts about upcoming maintenance updates or warnings about service instability. See below for nested schema.terminationProtection (boolean). Prevent service from being deleted. It is recommended to have this enabled for all services.userConfig (object). OpenSearch specific user configuration options. See below for nested schema.Appears on spec.
Authentication reference to Aiven token in a secret.
Required
key (string, MinLength: 1).name (string, MinLength: 1).Appears on spec.
Secret configuration.
Required
name (string, Immutable). Name of the secret resource to be created. By default, it is equal to the resource name.Optional
annotations (object, AdditionalProperties: string). Annotations added to the secret.labels (object, AdditionalProperties: string). Labels added to the secret.prefix (string). Prefix for the secret's keys. Added \"as is\" without any transformations. By default, is equal to the kind name in uppercase + underscore, e.g. KAFKA_, REDIS_, etc.Appears on spec.
ProjectVPCRef reference to ProjectVPC resource to use its ID as ProjectVPCID automatically.
Required
name (string, MinLength: 1).Optional
namespace (string, MinLength: 1).Appears on spec.
Service integrations to specify when creating a service. Not applied after initial service creation.
Required
integrationType (string, Enum: read_replica).sourceServiceName (string, MinLength: 1, MaxLength: 64).Appears on spec.
Defines the email addresses that will receive alerts about upcoming maintenance updates or warnings about service instability.
Required
email (string). Email address.Appears on spec.
OpenSearch specific user configuration options.
Optional
additional_backup_regions (array of strings, MaxItems: 1). Deprecated. Additional Cloud Regions for Backup Replication.ip_filter (array of objects, MaxItems: 1024). Allow incoming connections from CIDR address block, e.g. 10.20.0.0/16. See below for nested schema.private_access (object). Allow access to selected service ports from private networks. See below for nested schema.privatelink_access (object). Allow access to selected service components through Privatelink. See below for nested schema.project_to_fork_from (string, Immutable, Pattern: ^[a-z][-a-z0-9]{0,63}$|^$, MaxLength: 63). Name of another project to fork a service from. This has effect only when a new service is being created.public_access (object). Allow access to selected service ports from the public Internet. See below for nested schema.service_log (boolean). Store logs for the service so that they are available in the HTTP API and console.service_to_fork_from (string, Immutable, Pattern: ^[a-z][-a-z0-9]{0,63}$|^$, MaxLength: 64). Name of another service to fork from. This has effect only when a new service is being created.static_ips (boolean). Use static public IP addresses.Appears on spec.userConfig.
Allow incoming connections from CIDR address block, e.g. 10.20.0.0/16.
Required
network (string, MaxLength: 43). CIDR address block.Optional
description (string, MaxLength: 1024). Description for IP filter list entry.Appears on spec.userConfig.
Allow access to selected service ports from private networks.
Optional
clickhouse (boolean). Allow clients to connect to clickhouse with a DNS name that always resolves to the service's private IP addresses. Only available in certain network locations.clickhouse_https (boolean). Allow clients to connect to clickhouse_https with a DNS name that always resolves to the service's private IP addresses. Only available in certain network locations.clickhouse_mysql (boolean). Allow clients to connect to clickhouse_mysql with a DNS name that always resolves to the service's private IP addresses. Only available in certain network locations.prometheus (boolean). Allow clients to connect to prometheus with a DNS name that always resolves to the service's private IP addresses. Only available in certain network locations.Appears on spec.userConfig.
Allow access to selected service components through Privatelink.
Optional
clickhouse (boolean). Enable clickhouse.clickhouse_https (boolean). Enable clickhouse_https.clickhouse_mysql (boolean). Enable clickhouse_mysql.prometheus (boolean). Enable prometheus.Appears on spec.userConfig.
Allow access to selected service ports from the public Internet.
Optional
clickhouse (boolean). Allow clients to connect to clickhouse from the public internet for service nodes that are in a project VPC or another type of private network.clickhouse_https (boolean). Allow clients to connect to clickhouse_https from the public internet for service nodes that are in a project VPC or another type of private network.clickhouse_mysql (boolean). Allow clients to connect to clickhouse_mysql from the public internet for service nodes that are in a project VPC or another type of private network.prometheus (boolean). Allow clients to connect to prometheus from the public internet for service nodes that are in a project VPC or another type of private network.apiVersion: aiven.io/v1alpha1\nkind: ClickhouseDatabase\nmetadata:\n name: my-db\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n project: my-aiven-project\n serviceName: my-clickhouse\n databaseName: example-db\nInfo
To create this resource, a Secret containing Aiven token must be created first.
Apply the resource with:
kubectl apply -f example.yaml\nVerify the newly created ClickhouseDatabase:
kubectl get clickhousedatabases my-db\nThe output is similar to the following:
Name Database name Service Name Project \nmy-db example-db my-clickhouse my-aiven-project \nClickhouseDatabase is the Schema for the databases API.
Required
apiVersion (string). Value aiven.io/v1alpha1.kind (string). Value ClickhouseDatabase.metadata (object). Data that identifies the object, including a name string and optional namespace.spec (object). ClickhouseDatabaseSpec defines the desired state of ClickhouseDatabase. See below for nested schema.Appears on ClickhouseDatabase.
ClickhouseDatabaseSpec defines the desired state of ClickhouseDatabase.
Required
project (string, Immutable, Pattern: ^[a-zA-Z0-9_-]+$, MaxLength: 63). Identifies the project this resource belongs to.serviceName (string, Immutable, Pattern: ^[a-z][-a-z0-9]+$, MaxLength: 63). Specifies the name of the service that this resource belongs to.Optional
authSecretRef (object). Authentication reference to Aiven token in a secret. See below for nested schema.databaseName (string, Immutable, MaxLength: 63). Specifies the Clickhouse database name. Defaults to metadata.name if omitted. Note: metadata.name is ASCII-only. For UTF-8 names, use spec.databaseName, but ASCII is advised for compatibility.Appears on spec.
Authentication reference to Aiven token in a secret.
Required
key (string, MinLength: 1).name (string, MinLength: 1).apiVersion: aiven.io/v1alpha1\nkind: ClickhouseGrant\nmetadata:\n name: demo-ch-grant\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n project: my-aiven-project\n serviceName: my-clickhouse\n\n privilegeGrants:\n - grantees:\n - user: user1\n - user: my-clickhouse-user-\ud83e\udd84\n privileges:\n - SELECT\n - INSERT\n database: my-db\n # If table is omitted, the privileges are granted on all tables in the database\n # If columns is omitted, the privileges are granted on all columns in the table\n - grantees:\n - role: my-role\n privileges:\n - SELECT\n database: my-db\n table: my-table\n columns:\n - col1\n - col2\n\n roleGrants:\n - roles:\n - other-role\n grantees:\n - user: my-user\n - role: my-role\napiVersion: aiven.io/v1alpha1\nkind: ClickhouseGrant\nmetadata:\n name: my-clickhouse-grant\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n project: aiven-project-name\n serviceName: my-clickhouse-service\n\n privilegeGrants:\n - grantees:\n - role: my-clickhouse-role\n privileges:\n - INSERT\n - SELECT\n - CREATE TABLE\n - CREATE VIEW\n database: my-clickhouse-db\n roleGrants:\n - grantees:\n - user: my-clickhouse-user\n roles:\n - my-clickhouse-role\n\n---\n\napiVersion: aiven.io/v1alpha1\nkind: Clickhouse\nmetadata:\n name: my-clickhouse-service\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n project: aiven-project-name\n cloudName: google-europe-west1\n plan: startup-16\n\n---\n\napiVersion: aiven.io/v1alpha1\nkind: ClickhouseDatabase\nmetadata:\n name: my-clickhouse-db\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n project: aiven-project-name\n serviceName: my-clickhouse-service\n\n---\n\napiVersion: aiven.io/v1alpha1\nkind: ClickhouseUser\nmetadata:\n name: my-clickhouse-user\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n project: aiven-project-name\n serviceName: my-clickhouse-service\n\n---\n\napiVersion: aiven.io/v1alpha1\nkind: ClickhouseRole\nmetadata:\n name: my-clickhouse-role\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n project: aiven-project-name\n serviceName: my-clickhouse-service\n role: my-clickhouse-role\nInfo
To create this resource, a Secret containing Aiven token must be created first.
Apply the resource with:
kubectl apply -f example.yaml\nVerify the newly created ClickhouseGrant:
kubectl get clickhousegrants demo-ch-grant\nThe output is similar to the following:
Name Project Service Name \ndemo-ch-grant my-aiven-project my-clickhouse \nClickhouseGrant is the Schema for the ClickhouseGrants API
Ambiguity in the GRANT syntax
Due to an ambiguity in the GRANT syntax in Clickhouse, you should not have users and roles with the same name. It is not clear if a grant refers to the user or the role.
Required
apiVersion (string). Value aiven.io/v1alpha1.kind (string). Value ClickhouseGrant.metadata (object). Data that identifies the object, including a name string and optional namespace.spec (object). ClickhouseGrantSpec defines the desired state of ClickhouseGrant. See below for nested schema.Appears on ClickhouseGrant.
ClickhouseGrantSpec defines the desired state of ClickhouseGrant.
Required
project (string, Immutable, Pattern: ^[a-zA-Z0-9_-]+$, MaxLength: 63). Identifies the project this resource belongs to.serviceName (string, Immutable, Pattern: ^[a-z][-a-z0-9]+$, MaxLength: 63). Specifies the name of the service that this resource belongs to.Optional
authSecretRef (object). Authentication reference to Aiven token in a secret. See below for nested schema.privilegeGrants (array of objects). Configuration to grant a privilege. Privileges not in the manifest are revoked. Existing privileges are retained; new ones are granted. See below for nested schema.roleGrants (array of objects). Configuration to grant a role. Role grants not in the manifest are revoked. Existing role grants are retained; new ones are granted. See below for nested schema.Appears on spec.
Authentication reference to Aiven token in a secret.
Required
key (string, MinLength: 1).name (string, MinLength: 1).Appears on spec.
Configuration to grant a privilege. Privileges not in the manifest are revoked. Existing privileges are retained; new ones are granted.
Required
database (string). The database that the grant refers to.grantees (array of objects, MinItems: 1). List of grantees (users or roles) to grant the privilege to. See below for nested schema.privileges (array of strings). The privileges to grant, i.e. INSERT, SELECT. See https://clickhouse.com/docs/en/sql-reference/statements/grant#assigning-role-syntax.Optional
columns (array of strings). The column that the grant refers to.table (string). The tables that the grant refers to. To grant a privilege on all tables in a database, omit this field instead of writing table: \"*\".withGrantOption (boolean). If true, then the grantee (user or role) get the permission to execute the GRANT query. Users can grant privileges of the same scope they have and less. See https://clickhouse.com/docs/en/sql-reference/statements/grant#granting-privilege-syntax.Appears on spec.privilegeGrants.
List of grantees (users or roles) to grant the privilege to.
Optional
role (string).user (string).Appears on spec.
Configuration to grant a role. Role grants not in the manifest are revoked. Existing role grants are retained; new ones are granted.
Required
grantees (array of objects, MinItems: 1). List of grantees (users or roles) to grant the privilege to. See below for nested schema.roles (array of strings, MinItems: 1). List of roles to grant to the grantees.Optional
withAdminOption (boolean). If true, the grant is executed with ADMIN OPTION privilege. See https://clickhouse.com/docs/en/sql-reference/statements/grant#admin-option.Appears on spec.roleGrants.
List of grantees (users or roles) to grant the privilege to.
Optional
role (string).user (string).apiVersion: aiven.io/v1alpha1\nkind: ClickhouseRole\nmetadata:\n name: my-role\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n project: my-aiven-project\n serviceName: my-clickhouse\n role: my-role\nInfo
To create this resource, a Secret containing Aiven token must be created first.
Apply the resource with:
kubectl apply -f example.yaml\nVerify the newly created ClickhouseRole:
kubectl get clickhouseroles my-role\nThe output is similar to the following:
Name Project Service Name Role \nmy-role my-aiven-project my-clickhouse my-role \nClickhouseRole is the Schema for the clickhouseroles API.
Required
apiVersion (string). Value aiven.io/v1alpha1.kind (string). Value ClickhouseRole.metadata (object). Data that identifies the object, including a name string and optional namespace.spec (object). ClickhouseRoleSpec defines the desired state of ClickhouseRole. See below for nested schema.Appears on ClickhouseRole.
ClickhouseRoleSpec defines the desired state of ClickhouseRole.
Required
project (string, Immutable, Pattern: ^[a-zA-Z0-9_-]+$, MaxLength: 63). Identifies the project this resource belongs to.role (string, Immutable, MaxLength: 255). The role that is to be created.serviceName (string, Immutable, Pattern: ^[a-z][-a-z0-9]+$, MaxLength: 63). Specifies the name of the service that this resource belongs to.Optional
authSecretRef (object). Authentication reference to Aiven token in a secret. See below for nested schema.Appears on spec.
Authentication reference to Aiven token in a secret.
Required
key (string, MinLength: 1).name (string, MinLength: 1).apiVersion: aiven.io/v1alpha1\nkind: ClickhouseUser\nmetadata:\n name: my-clickhouse-user\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n connInfoSecretTarget:\n name: clickhouse-user-secret\n annotations:\n foo: bar\n labels:\n baz: egg\n\n project: my-aiven-project\n serviceName: my-clickhouse\n username: example-username\n\n---\n\napiVersion: aiven.io/v1alpha1\nkind: Clickhouse\nmetadata:\n name: my-clickhouse\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n project: my-aiven-project\n cloudName: google-europe-west1\n plan: startup-16\nInfo
To create this resource, a Secret containing Aiven token must be created first.
Apply the resource with:
kubectl apply -f example.yaml\nVerify the newly created ClickhouseUser:
kubectl get clickhouseusers my-clickhouse-user\nThe output is similar to the following:
Name Username Service Name Project \nmy-clickhouse-user example-username my-clickhouse my-aiven-project \nTo view the details of the Secret, use the following command:
kubectl describe secret clickhouse-user-secret\nYou can use the jq to quickly decode the Secret:
kubectl get secret clickhouse-user-secret -o json | jq '.data | map_values(@base64d)'\nThe output is similar to the following:
{\n \"CLICKHOUSEUSER_HOST\": \"<secret>\",\n \"CLICKHOUSEUSER_PORT\": \"<secret>\",\n \"CLICKHOUSEUSER_USER\": \"<secret>\",\n \"CLICKHOUSEUSER_PASSWORD\": \"<secret>\",\n}\nClickhouseUser is the Schema for the clickhouseusers API.
Exposes secret keys
CLICKHOUSEUSER_HOST, CLICKHOUSEUSER_PORT, CLICKHOUSEUSER_USER, CLICKHOUSEUSER_PASSWORD.
Required
apiVersion (string). Value aiven.io/v1alpha1.kind (string). Value ClickhouseUser.metadata (object). Data that identifies the object, including a name string and optional namespace.spec (object). ClickhouseUserSpec defines the desired state of ClickhouseUser. See below for nested schema.Appears on ClickhouseUser.
ClickhouseUserSpec defines the desired state of ClickhouseUser.
Required
project (string, Immutable, Pattern: ^[a-zA-Z0-9_-]+$, MaxLength: 63). Identifies the project this resource belongs to.serviceName (string, Immutable, Pattern: ^[a-z][-a-z0-9]+$, MaxLength: 63). Specifies the name of the service that this resource belongs to.Optional
authSecretRef (object). Authentication reference to Aiven token in a secret. See below for nested schema.connInfoSecretTarget (object). Secret configuration. See below for nested schema.connInfoSecretTargetDisabled (boolean, Immutable). When true, the secret containing connection information will not be created, defaults to false. This field cannot be changed after resource creation.username (string, Immutable, MaxLength: 63). Name of the Clickhouse user. Defaults to metadata.name if omitted. Note: metadata.name is ASCII-only. For UTF-8 names, use spec.username, but ASCII is advised for compatibility.Appears on spec.
Authentication reference to Aiven token in a secret.
Required
key (string, MinLength: 1).name (string, MinLength: 1).Appears on spec.
Secret configuration.
Required
name (string, Immutable). Name of the secret resource to be created. By default, it is equal to the resource name.Optional
annotations (object, AdditionalProperties: string). Annotations added to the secret.labels (object, AdditionalProperties: string). Labels added to the secret.prefix (string). Prefix for the secret's keys. Added \"as is\" without any transformations. By default, is equal to the kind name in uppercase + underscore, e.g. KAFKA_, REDIS_, etc.apiVersion: aiven.io/v1alpha1\nkind: ConnectionPool\nmetadata:\n name: my-connection-pool\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n project: aiven-project-name\n serviceName: my-pg\n databaseName: my-database\n username: my-service-user\n poolMode: transaction\n poolSize: 25\n\n---\n\napiVersion: aiven.io/v1alpha1\nkind: PostgreSQL\nmetadata:\n name: my-pg\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n project: aiven-project-name\n cloudName: google-europe-west1\n plan: startup-4\n\n---\n\napiVersion: aiven.io/v1alpha1\nkind: Database\nmetadata:\n name: my-database\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n project: aiven-project-name\n serviceName: my-pg\n\n---\n\napiVersion: aiven.io/v1alpha1\nkind: ServiceUser\nmetadata:\n name: my-service-user\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n project: aiven-project-name\n serviceName: my-pg\nInfo
To create this resource, a Secret containing Aiven token must be created first.
Apply the resource with:
kubectl apply -f example.yaml\nVerify the newly created ConnectionPool:
kubectl get connectionpools my-connection-pool\nThe output is similar to the following:
Name Service Name Project Database Username Pool Size Pool Mode \nmy-connection-pool my-pg aiven-project-name my-database my-service-user 25 transaction \nConnectionPool is the Schema for the connectionpools API.
Exposes secret keys
CONNECTIONPOOL_NAME, CONNECTIONPOOL_HOST, CONNECTIONPOOL_PORT, CONNECTIONPOOL_DATABASE, CONNECTIONPOOL_USER, CONNECTIONPOOL_PASSWORD, CONNECTIONPOOL_SSLMODE, CONNECTIONPOOL_DATABASE_URI, CONNECTIONPOOL_CA_CERT.
Required
apiVersion (string). Value aiven.io/v1alpha1.kind (string). Value ConnectionPool.metadata (object). Data that identifies the object, including a name string and optional namespace.spec (object). ConnectionPoolSpec defines the desired state of ConnectionPool. See below for nested schema.Appears on ConnectionPool.
ConnectionPoolSpec defines the desired state of ConnectionPool.
Required
databaseName (string, MaxLength: 40). Name of the database the pool connects to.project (string, Immutable, Pattern: ^[a-zA-Z0-9_-]+$, MaxLength: 63). Identifies the project this resource belongs to.serviceName (string, Immutable, Pattern: ^[a-z][-a-z0-9]+$, MaxLength: 63). Specifies the name of the service that this resource belongs to.username (string, MaxLength: 64). Name of the service user used to connect to the database.Optional
authSecretRef (object). Authentication reference to Aiven token in a secret. See below for nested schema.connInfoSecretTarget (object). Secret configuration. See below for nested schema.connInfoSecretTargetDisabled (boolean, Immutable). When true, the secret containing connection information will not be created, defaults to false. This field cannot be changed after resource creation.poolMode (string, Enum: session, transaction, statement). Mode the pool operates in (session, transaction, statement).poolSize (integer). Number of connections the pool may create towards the backend server.Appears on spec.
Authentication reference to Aiven token in a secret.
Required
key (string, MinLength: 1).name (string, MinLength: 1).Appears on spec.
Secret configuration.
Required
name (string, Immutable). Name of the secret resource to be created. By default, it is equal to the resource name.Optional
annotations (object, AdditionalProperties: string). Annotations added to the secret.labels (object, AdditionalProperties: string). Labels added to the secret.prefix (string). Prefix for the secret's keys. Added \"as is\" without any transformations. By default, is equal to the kind name in uppercase + underscore, e.g. KAFKA_, REDIS_, etc.apiVersion: aiven.io/v1alpha1\nkind: Database\nmetadata:\n name: my-db\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n project: aiven-project-name\n serviceName: my-service\n\n lcCtype: en_US.UTF-8\n lcCollate: en_US.UTF-8\nInfo
To create this resource, a Secret containing Aiven token must be created first.
Apply the resource with:
kubectl apply -f example.yaml\nVerify the newly created Database:
kubectl get databases my-db\nThe output is similar to the following:
Name Project Service Name \nmy-db aiven-project-name my-service \nDatabase is the Schema for the databases API.
Required
apiVersion (string). Value aiven.io/v1alpha1.kind (string). Value Database.metadata (object). Data that identifies the object, including a name string and optional namespace.spec (object). DatabaseSpec defines the desired state of Database. See below for nested schema.Appears on Database.
DatabaseSpec defines the desired state of Database.
Required
project (string, Immutable, Pattern: ^[a-zA-Z0-9_-]+$, MaxLength: 63). Identifies the project this resource belongs to.serviceName (string, Immutable, Pattern: ^[a-z][-a-z0-9]+$, MaxLength: 63). Specifies the name of the service that this resource belongs to.Optional
authSecretRef (object). Authentication reference to Aiven token in a secret. See below for nested schema.lcCollate (string, Immutable, MaxLength: 128). Default string sort order (LC_COLLATE) of the database. Default value: en_US.UTF-8.lcCtype (string, Immutable, MaxLength: 128). Default character classification (LC_CTYPE) of the database. Default value: en_US.UTF-8.terminationProtection (boolean). It is a Kubernetes side deletion protections, which prevents the database from being deleted by Kubernetes. It is recommended to enable this for any production databases containing critical data.Appears on spec.
Authentication reference to Aiven token in a secret.
Required
key (string, MinLength: 1).name (string, MinLength: 1).apiVersion: aiven.io/v1alpha1\nkind: Grafana\nmetadata:\n name: my-grafana\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n connInfoSecretTarget:\n name: grafana-secret\n prefix: MY_SECRET_PREFIX_\n annotations:\n foo: bar\n labels:\n baz: egg\n\n project: my-aiven-project\n cloudName: google-europe-west1\n plan: startup-1\n\n maintenanceWindowDow: sunday\n maintenanceWindowTime: 11:00:00\n\n userConfig:\n public_access:\n grafana: true\n ip_filter:\n - network: 0.0.0.0\n description: whatever\n - network: 10.20.0.0/16\nInfo
To create this resource, a Secret containing Aiven token must be created first.
Apply the resource with:
kubectl apply -f example.yaml\nVerify the newly created Grafana:
kubectl get grafanas my-grafana\nThe output is similar to the following:
Name Project Region Plan State \nmy-grafana my-aiven-project google-europe-west1 startup-1 RUNNING \nTo view the details of the Secret, use the following command:
kubectl describe secret grafana-secret\nYou can use the jq to quickly decode the Secret:
kubectl get secret grafana-secret -o json | jq '.data | map_values(@base64d)'\nThe output is similar to the following:
{\n \"GRAFANA_HOST\": \"<secret>\",\n \"GRAFANA_PORT\": \"<secret>\",\n \"GRAFANA_USER\": \"<secret>\",\n \"GRAFANA_PASSWORD\": \"<secret>\",\n \"GRAFANA_URI\": \"<secret>\",\n \"GRAFANA_HOSTS\": \"<secret>\",\n}\nGrafana is the Schema for the grafanas API.
Exposes secret keys
GRAFANA_HOST, GRAFANA_PORT, GRAFANA_USER, GRAFANA_PASSWORD, GRAFANA_URI, GRAFANA_HOSTS.
Required
apiVersion (string). Value aiven.io/v1alpha1.kind (string). Value Grafana.metadata (object). Data that identifies the object, including a name string and optional namespace.spec (object). GrafanaSpec defines the desired state of Grafana. See below for nested schema.Appears on Grafana.
GrafanaSpec defines the desired state of Grafana.
Required
plan (string, MaxLength: 128). Subscription plan.project (string, Immutable, Pattern: ^[a-zA-Z0-9_-]+$, MaxLength: 63). Identifies the project this resource belongs to.Optional
authSecretRef (object). Authentication reference to Aiven token in a secret. See below for nested schema.cloudName (string, MaxLength: 256). Cloud the service runs in.connInfoSecretTarget (object). Secret configuration. See below for nested schema.connInfoSecretTargetDisabled (boolean, Immutable). When true, the secret containing connection information will not be created, defaults to false. This field cannot be changed after resource creation.disk_space (string, Pattern: (?i)^[1-9][0-9]*(GiB|G)?$). The disk space of the service, possible values depend on the service type, the cloud provider and the project. Reducing will result in the service re-balancing. The removal of this field does not change the value.maintenanceWindowDow (string, Enum: monday, tuesday, wednesday, thursday, friday, saturday, sunday). Day of week when maintenance operations should be performed. One monday, tuesday, wednesday, etc.maintenanceWindowTime (string, MaxLength: 8). Time of day when maintenance operations should be performed. UTC time in HH:mm:ss format.projectVPCRef (object). ProjectVPCRef reference to ProjectVPC resource to use its ID as ProjectVPCID automatically. See below for nested schema.projectVpcId (string, MaxLength: 36). Identifier of the VPC the service should be in, if any.serviceIntegrations (array of objects, Immutable, MaxItems: 1). Service integrations to specify when creating a service. Not applied after initial service creation. See below for nested schema.tags (object, AdditionalProperties: string). Tags are key-value pairs that allow you to categorize services.technicalEmails (array of objects, MaxItems: 10). Defines the email addresses that will receive alerts about upcoming maintenance updates or warnings about service instability. See below for nested schema.terminationProtection (boolean). Prevent service from being deleted. It is recommended to have this enabled for all services.userConfig (object). Cassandra specific user configuration options. See below for nested schema.Appears on spec.
Authentication reference to Aiven token in a secret.
Required
key (string, MinLength: 1).name (string, MinLength: 1).Appears on spec.
Secret configuration.
Required
name (string, Immutable). Name of the secret resource to be created. By default, it is equal to the resource name.Optional
annotations (object, AdditionalProperties: string). Annotations added to the secret.labels (object, AdditionalProperties: string). Labels added to the secret.prefix (string). Prefix for the secret's keys. Added \"as is\" without any transformations. By default, is equal to the kind name in uppercase + underscore, e.g. KAFKA_, REDIS_, etc.Appears on spec.
ProjectVPCRef reference to ProjectVPC resource to use its ID as ProjectVPCID automatically.
Required
name (string, MinLength: 1).Optional
namespace (string, MinLength: 1).Appears on spec.
Service integrations to specify when creating a service. Not applied after initial service creation.
Required
integrationType (string, Enum: read_replica).sourceServiceName (string, MinLength: 1, MaxLength: 64).Appears on spec.
Defines the email addresses that will receive alerts about upcoming maintenance updates or warnings about service instability.
Required
email (string). Email address.Appears on spec.
Cassandra specific user configuration options.
Optional
additional_backup_regions (array of strings, MaxItems: 1). Additional Cloud Regions for Backup Replication.alerting_enabled (boolean). Enable or disable Grafana legacy alerting functionality. This should not be enabled with unified_alerting_enabled.alerting_error_or_timeout (string, Enum: alerting, keep_state). Default error or timeout setting for new alerting rules.alerting_max_annotations_to_keep (integer, Minimum: 0, Maximum: 1000000). Max number of alert annotations that Grafana stores. 0 (default) keeps all alert annotations.alerting_nodata_or_nullvalues (string, Enum: alerting, no_data, keep_state, ok). Default value for 'no data or null values' for new alerting rules.allow_embedding (boolean). Allow embedding Grafana dashboards with iframe/frame/object/embed tags. Disabled by default to limit impact of clickjacking.auth_azuread (object). Azure AD OAuth integration. See below for nested schema.auth_basic_enabled (boolean). Enable or disable basic authentication form, used by Grafana built-in login.auth_generic_oauth (object). Generic OAuth integration. See below for nested schema.auth_github (object). Github Auth integration. See below for nested schema.auth_gitlab (object). GitLab Auth integration. See below for nested schema.auth_google (object). Google Auth integration. See below for nested schema.cookie_samesite (string, Enum: lax, strict, none). Cookie SameSite attribute: strict prevents sending cookie for cross-site requests, effectively disabling direct linking from other sites to Grafana. lax is the default value.custom_domain (string, MaxLength: 255). Serve the web frontend using a custom CNAME pointing to the Aiven DNS name.dashboard_previews_enabled (boolean). This feature is new in Grafana 9 and is quite resource intensive. It may cause low-end plans to work more slowly while the dashboard previews are rendering.dashboards_min_refresh_interval (string, Pattern: ^[0-9]+(ms|s|m|h|d)$, MaxLength: 16). Signed sequence of decimal numbers, followed by a unit suffix (ms, s, m, h, d), e.g. 30s, 1h.dashboards_versions_to_keep (integer, Minimum: 1, Maximum: 100). Dashboard versions to keep per dashboard.dataproxy_send_user_header (boolean). Send X-Grafana-User header to data source.dataproxy_timeout (integer, Minimum: 15, Maximum: 90). Timeout for data proxy requests in seconds.date_formats (object). Grafana date format specifications. See below for nested schema.disable_gravatar (boolean). Set to true to disable gravatar. Defaults to false (gravatar is enabled).editors_can_admin (boolean). Editors can manage folders, teams and dashboards created by them.external_image_storage (object). External image store settings. See below for nested schema.google_analytics_ua_id (string, Pattern: ^(G|UA|YT|MO)-[a-zA-Z0-9-]+$, MaxLength: 64). Google Analytics ID.ip_filter (array of objects, MaxItems: 1024). Allow incoming connections from CIDR address block, e.g. 10.20.0.0/16. See below for nested schema.metrics_enabled (boolean). Enable Grafana /metrics endpoint.oauth_allow_insecure_email_lookup (boolean). Enforce user lookup based on email instead of the unique ID provided by the IdP.private_access (object). Allow access to selected service ports from private networks. See below for nested schema.privatelink_access (object). Allow access to selected service components through Privatelink. See below for nested schema.project_to_fork_from (string, Immutable, Pattern: ^[a-z][-a-z0-9]{0,63}$|^$, MaxLength: 63). Name of another project to fork a service from. This has effect only when a new service is being created.public_access (object). Allow access to selected service ports from the public Internet. See below for nested schema.recovery_basebackup_name (string, Pattern: ^[a-zA-Z0-9-_:.]+$, MaxLength: 128). Name of the basebackup to restore in forked service.service_log (boolean). Store logs for the service so that they are available in the HTTP API and console.service_to_fork_from (string, Immutable, Pattern: ^[a-z][-a-z0-9]{0,63}$|^$, MaxLength: 64). Name of another service to fork from. This has effect only when a new service is being created.smtp_server (object). SMTP server settings. See below for nested schema.static_ips (boolean). Use static public IP addresses.unified_alerting_enabled (boolean). Enable or disable Grafana unified alerting functionality. By default this is enabled and any legacy alerts will be migrated on upgrade to Grafana 9+. To stay on legacy alerting, set unified_alerting_enabled to false and alerting_enabled to true. See https://grafana.com/docs/grafana/latest/alerting/set-up/migrating-alerts/ for more details.user_auto_assign_org (boolean). Auto-assign new users on signup to main organization. Defaults to false.user_auto_assign_org_role (string, Enum: Viewer, Admin, Editor). Set role for new signups. Defaults to Viewer.viewers_can_edit (boolean). Users with view-only permission can edit but not save dashboards.Appears on spec.userConfig.
Azure AD OAuth integration.
Required
auth_url (string, MaxLength: 2048). Authorization URL.client_id (string, Pattern: ^[\\040-\\176]+$, MaxLength: 1024). Client ID from provider.client_secret (string, Pattern: ^[\\040-\\176]+$, MaxLength: 1024). Client secret from provider.token_url (string, MaxLength: 2048). Token URL.Optional
allow_sign_up (boolean). Automatically sign-up users on successful sign-in.allowed_domains (array of strings, MaxItems: 50). Allowed domains.allowed_groups (array of strings, MaxItems: 50). Require users to belong to one of given groups.Appears on spec.userConfig.
Generic OAuth integration.
Required
api_url (string, MaxLength: 2048). API URL.auth_url (string, MaxLength: 2048). Authorization URL.client_id (string, Pattern: ^[\\040-\\176]+$, MaxLength: 1024). Client ID from provider.client_secret (string, Pattern: ^[\\040-\\176]+$, MaxLength: 1024). Client secret from provider.token_url (string, MaxLength: 2048). Token URL.Optional
allow_sign_up (boolean). Automatically sign-up users on successful sign-in.allowed_domains (array of strings, MaxItems: 50). Allowed domains.allowed_organizations (array of strings, MaxItems: 50). Require user to be member of one of the listed organizations.auto_login (boolean). Allow users to bypass the login screen and automatically log in.name (string, Pattern: ^[a-zA-Z0-9_\\- ]+$, MaxLength: 128). Name of the OAuth integration.scopes (array of strings, MaxItems: 50). OAuth scopes.Appears on spec.userConfig.
Github Auth integration.
Required
client_id (string, Pattern: ^[\\040-\\176]+$, MaxLength: 1024). Client ID from provider.client_secret (string, Pattern: ^[\\040-\\176]+$, MaxLength: 1024). Client secret from provider.Optional
allow_sign_up (boolean). Automatically sign-up users on successful sign-in.allowed_organizations (array of strings, MaxItems: 50). Require users to belong to one of given organizations.auto_login (boolean). Allow users to bypass the login screen and automatically log in.skip_org_role_sync (boolean). Stop automatically syncing user roles.team_ids (array of integers, MaxItems: 50). Require users to belong to one of given team IDs.Appears on spec.userConfig.
GitLab Auth integration.
Required
allowed_groups (array of strings, MaxItems: 50). Require users to belong to one of given groups.client_id (string, Pattern: ^[\\040-\\176]+$, MaxLength: 1024). Client ID from provider.client_secret (string, Pattern: ^[\\040-\\176]+$, MaxLength: 1024). Client secret from provider.Optional
allow_sign_up (boolean). Automatically sign-up users on successful sign-in.api_url (string, MaxLength: 2048). API URL. This only needs to be set when using self hosted GitLab.auth_url (string, MaxLength: 2048). Authorization URL. This only needs to be set when using self hosted GitLab.token_url (string, MaxLength: 2048). Token URL. This only needs to be set when using self hosted GitLab.Appears on spec.userConfig.
Google Auth integration.
Required
allowed_domains (array of strings, MaxItems: 64). Domains allowed to sign-in to this Grafana.client_id (string, Pattern: ^[\\040-\\176]+$, MaxLength: 1024). Client ID from provider.client_secret (string, Pattern: ^[\\040-\\176]+$, MaxLength: 1024). Client secret from provider.Optional
allow_sign_up (boolean). Automatically sign-up users on successful sign-in.Appears on spec.userConfig.
Grafana date format specifications.
Optional
default_timezone (string, MaxLength: 64). Default time zone for user preferences. Value browser uses browser local time zone.full_date (string, MaxLength: 128). Moment.js style format string for cases where full date is shown.interval_day (string, MaxLength: 128). Moment.js style format string used when a time requiring day accuracy is shown.interval_hour (string, MaxLength: 128). Moment.js style format string used when a time requiring hour accuracy is shown.interval_minute (string, MaxLength: 128). Moment.js style format string used when a time requiring minute accuracy is shown.interval_month (string, MaxLength: 128). Moment.js style format string used when a time requiring month accuracy is shown.interval_second (string, MaxLength: 128). Moment.js style format string used when a time requiring second accuracy is shown.interval_year (string, MaxLength: 128). Moment.js style format string used when a time requiring year accuracy is shown.Appears on spec.userConfig.
External image store settings.
Required
access_key (string, Pattern: ^[A-Z0-9]+$, MaxLength: 4096). S3 access key. Requires permissions to the S3 bucket for the s3:PutObject and s3:PutObjectAcl actions.bucket_url (string, MaxLength: 2048). Bucket URL for S3.provider (string, Enum: s3). Provider type.secret_key (string, Pattern: ^[A-Za-z0-9/+=]+$, MaxLength: 4096). S3 secret key.Appears on spec.userConfig.
Allow incoming connections from CIDR address block, e.g. 10.20.0.0/16.
Required
network (string, MaxLength: 43). CIDR address block.Optional
description (string, MaxLength: 1024). Description for IP filter list entry.Appears on spec.userConfig.
Allow access to selected service ports from private networks.
Required
grafana (boolean). Allow clients to connect to grafana with a DNS name that always resolves to the service's private IP addresses. Only available in certain network locations.Appears on spec.userConfig.
Allow access to selected service components through Privatelink.
Required
grafana (boolean). Enable grafana.Appears on spec.userConfig.
Allow access to selected service ports from the public Internet.
Required
grafana (boolean). Allow clients to connect to grafana from the public internet for service nodes that are in a project VPC or another type of private network.Appears on spec.userConfig.
SMTP server settings.
Required
from_address (string, MaxLength: 319). Address used for sending emails.host (string, MaxLength: 255). Server hostname or IP.port (integer, Minimum: 1, Maximum: 65535). SMTP server port.Optional
from_name (string, Pattern: ^[^\\x00-\\x1F]+$, MaxLength: 128). Name used in outgoing emails, defaults to Grafana.password (string, Pattern: ^[^\\x00-\\x1F]+$, MaxLength: 255). Password for SMTP authentication.skip_verify (boolean). Skip verifying server certificate. Defaults to false.starttls_policy (string, Enum: OpportunisticStartTLS, MandatoryStartTLS, NoStartTLS). Either OpportunisticStartTLS, MandatoryStartTLS or NoStartTLS. Default is OpportunisticStartTLS.username (string, Pattern: ^[^\\x00-\\x1F]+$, MaxLength: 255). Username for SMTP authentication.apiVersion: aiven.io/v1alpha1\nkind: Kafka\nmetadata:\n name: my-kafka\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n connInfoSecretTarget:\n name: kafka-secret\n prefix: MY_SECRET_PREFIX_\n annotations:\n foo: bar\n labels:\n baz: egg\n\n project: my-aiven-project\n cloudName: google-europe-west1\n plan: startup-2\n\n maintenanceWindowDow: friday\n maintenanceWindowTime: 23:00:00\nInfo
To create this resource, a Secret containing Aiven token must be created first.
Apply the resource with:
kubectl apply -f example.yaml\nVerify the newly created Kafka:
kubectl get kafkas my-kafka\nThe output is similar to the following:
Name Project Region Plan State \nmy-kafka my-aiven-project google-europe-west1 startup-2 RUNNING \nTo view the details of the Secret, use the following command:
kubectl describe secret kafka-secret\nYou can use the jq to quickly decode the Secret:
kubectl get secret kafka-secret -o json | jq '.data | map_values(@base64d)'\nThe output is similar to the following:
{\n \"KAFKA_HOST\": \"<secret>\",\n \"KAFKA_PORT\": \"<secret>\",\n \"KAFKA_USERNAME\": \"<secret>\",\n \"KAFKA_PASSWORD\": \"<secret>\",\n \"KAFKA_ACCESS_CERT\": \"<secret>\",\n \"KAFKA_ACCESS_KEY\": \"<secret>\",\n \"KAFKA_SASL_HOST\": \"<secret>\",\n \"KAFKA_SASL_PORT\": \"<secret>\",\n \"KAFKA_SCHEMA_REGISTRY_HOST\": \"<secret>\",\n \"KAFKA_SCHEMA_REGISTRY_PORT\": \"<secret>\",\n \"KAFKA_CONNECT_HOST\": \"<secret>\",\n \"KAFKA_CONNECT_PORT\": \"<secret>\",\n \"KAFKA_REST_HOST\": \"<secret>\",\n \"KAFKA_REST_PORT\": \"<secret>\",\n \"KAFKA_CA_CERT\": \"<secret>\",\n}\nKafka is the Schema for the kafkas API.
Exposes secret keys
KAFKA_HOST, KAFKA_PORT, KAFKA_USERNAME, KAFKA_PASSWORD, KAFKA_ACCESS_CERT, KAFKA_ACCESS_KEY, KAFKA_SASL_HOST, KAFKA_SASL_PORT, KAFKA_SCHEMA_REGISTRY_HOST, KAFKA_SCHEMA_REGISTRY_PORT, KAFKA_CONNECT_HOST, KAFKA_CONNECT_PORT, KAFKA_REST_HOST, KAFKA_REST_PORT, KAFKA_CA_CERT.
Required
apiVersion (string). Value aiven.io/v1alpha1.kind (string). Value Kafka.metadata (object). Data that identifies the object, including a name string and optional namespace.spec (object). KafkaSpec defines the desired state of Kafka. See below for nested schema.Appears on Kafka.
KafkaSpec defines the desired state of Kafka.
Required
plan (string, MaxLength: 128). Subscription plan.project (string, Immutable, Pattern: ^[a-zA-Z0-9_-]+$, MaxLength: 63). Identifies the project this resource belongs to.Optional
authSecretRef (object). Authentication reference to Aiven token in a secret. See below for nested schema.cloudName (string, MaxLength: 256). Cloud the service runs in.connInfoSecretTarget (object). Secret configuration. See below for nested schema.connInfoSecretTargetDisabled (boolean, Immutable). When true, the secret containing connection information will not be created, defaults to false. This field cannot be changed after resource creation.disk_space (string, Pattern: (?i)^[1-9][0-9]*(GiB|G)?$). The disk space of the service, possible values depend on the service type, the cloud provider and the project. Reducing will result in the service re-balancing. The removal of this field does not change the value.karapace (boolean). Switch the service to use Karapace for schema registry and REST proxy.maintenanceWindowDow (string, Enum: monday, tuesday, wednesday, thursday, friday, saturday, sunday). Day of week when maintenance operations should be performed. One monday, tuesday, wednesday, etc.maintenanceWindowTime (string, MaxLength: 8). Time of day when maintenance operations should be performed. UTC time in HH:mm:ss format.projectVPCRef (object). ProjectVPCRef reference to ProjectVPC resource to use its ID as ProjectVPCID automatically. See below for nested schema.projectVpcId (string, MaxLength: 36). Identifier of the VPC the service should be in, if any.serviceIntegrations (array of objects, Immutable, MaxItems: 1). Service integrations to specify when creating a service. Not applied after initial service creation. See below for nested schema.tags (object, AdditionalProperties: string). Tags are key-value pairs that allow you to categorize services.technicalEmails (array of objects, MaxItems: 10). Defines the email addresses that will receive alerts about upcoming maintenance updates or warnings about service instability. See below for nested schema.terminationProtection (boolean). Prevent service from being deleted. It is recommended to have this enabled for all services.userConfig (object). Kafka specific user configuration options. See below for nested schema.Appears on spec.
Authentication reference to Aiven token in a secret.
Required
key (string, MinLength: 1).name (string, MinLength: 1).Appears on spec.
Secret configuration.
Required
name (string, Immutable). Name of the secret resource to be created. By default, it is equal to the resource name.Optional
annotations (object, AdditionalProperties: string). Annotations added to the secret.labels (object, AdditionalProperties: string). Labels added to the secret.prefix (string). Prefix for the secret's keys. Added \"as is\" without any transformations. By default, is equal to the kind name in uppercase + underscore, e.g. KAFKA_, REDIS_, etc.Appears on spec.
ProjectVPCRef reference to ProjectVPC resource to use its ID as ProjectVPCID automatically.
Required
name (string, MinLength: 1).Optional
namespace (string, MinLength: 1).Appears on spec.
Service integrations to specify when creating a service. Not applied after initial service creation.
Required
integrationType (string, Enum: read_replica).sourceServiceName (string, MinLength: 1, MaxLength: 64).Appears on spec.
Defines the email addresses that will receive alerts about upcoming maintenance updates or warnings about service instability.
Required
email (string). Email address.Appears on spec.
Kafka specific user configuration options.
Optional
additional_backup_regions (array of strings, MaxItems: 1). Deprecated. Additional Cloud Regions for Backup Replication.aiven_kafka_topic_messages (boolean). Allow access to read Kafka topic messages in the Aiven Console and REST API.custom_domain (string, MaxLength: 255). Serve the web frontend using a custom CNAME pointing to the Aiven DNS name.ip_filter (array of objects, MaxItems: 1024). Allow incoming connections from CIDR address block, e.g. 10.20.0.0/16. See below for nested schema.kafka (object). Kafka broker configuration values. See below for nested schema.kafka_authentication_methods (object). Kafka authentication methods. See below for nested schema.kafka_connect (boolean). Enable Kafka Connect service.kafka_connect_config (object). Kafka Connect configuration values. See below for nested schema.kafka_connect_secret_providers. See below for } (array of objects). Configure external secret providers in order to reference external secrets in connector configuration. Currently Hashicorp Vault (provider: vault, auth_method: token) and AWS Secrets Manager (provider: aws, auth_method: credentials) are supported. Secrets can be referenced in connector config with ${::nested schema. kafka_rest (boolean). Enable Kafka-REST service.kafka_rest_authorization (boolean). Enable authorization in Kafka-REST service.kafka_rest_config (object). Kafka REST configuration. See below for nested schema.kafka_version (string, Enum: 3.4, 3.5, 3.6, 3.7). Kafka major version.letsencrypt_sasl_privatelink (boolean). Use Letsencrypt CA for Kafka SASL via Privatelink.private_access (object). Allow access to selected service ports from private networks. See below for nested schema.privatelink_access (object). Allow access to selected service components through Privatelink. See below for nested schema.public_access (object). Allow access to selected service ports from the public Internet. See below for nested schema.schema_registry (boolean). Enable Schema-Registry service.schema_registry_config (object). Schema Registry configuration. See below for nested schema.service_log (boolean). Store logs for the service so that they are available in the HTTP API and console.static_ips (boolean). Use static public IP addresses.tiered_storage (object). Tiered storage configuration. See below for nested schema.Appears on spec.userConfig.
Allow incoming connections from CIDR address block, e.g. 10.20.0.0/16.
Required
network (string, MaxLength: 43). CIDR address block.Optional
description (string, MaxLength: 1024). Description for IP filter list entry.Appears on spec.userConfig.
Kafka broker configuration values.
Optional
auto_create_topics_enable (boolean). Enable auto creation of topics.compression_type (string, Enum: gzip, snappy, lz4, zstd, uncompressed, producer). Specify the final compression type for a given topic. This configuration accepts the standard compression codecs (gzip, snappy, lz4, zstd). It additionally accepts uncompressed which is equivalent to no compression; and producer which means retain the original compression codec set by the producer.connections_max_idle_ms (integer, Minimum: 1000, Maximum: 3600000). Idle connections timeout: the server socket processor threads close the connections that idle for longer than this.default_replication_factor (integer, Minimum: 1, Maximum: 10). Replication factor for autocreated topics.group_initial_rebalance_delay_ms (integer, Minimum: 0, Maximum: 300000). The amount of time, in milliseconds, the group coordinator will wait for more consumers to join a new group before performing the first rebalance. A longer delay means potentially fewer rebalances, but increases the time until processing begins. The default value for this is 3 seconds. During development and testing it might be desirable to set this to 0 in order to not delay test execution time.group_max_session_timeout_ms (integer, Minimum: 0, Maximum: 1800000). The maximum allowed session timeout for registered consumers. Longer timeouts give consumers more time to process messages in between heartbeats at the cost of a longer time to detect failures.group_min_session_timeout_ms (integer, Minimum: 0, Maximum: 60000). The minimum allowed session timeout for registered consumers. Longer timeouts give consumers more time to process messages in between heartbeats at the cost of a longer time to detect failures.log_cleaner_delete_retention_ms (integer, Minimum: 0, Maximum: 315569260000). How long are delete records retained?.log_cleaner_max_compaction_lag_ms (integer, Minimum: 30000). The maximum amount of time message will remain uncompacted. Only applicable for logs that are being compacted.log_cleaner_min_cleanable_ratio (number, Minimum: 0.2, Maximum: 0.9). Controls log compactor frequency. Larger value means more frequent compactions but also more space wasted for logs. Consider setting log.cleaner.max.compaction.lag.ms to enforce compactions sooner, instead of setting a very high value for this option.log_cleaner_min_compaction_lag_ms (integer, Minimum: 0). The minimum time a message will remain uncompacted in the log. Only applicable for logs that are being compacted.log_cleanup_policy (string, Enum: delete, compact, compact,delete). The default cleanup policy for segments beyond the retention window.log_flush_interval_messages (integer, Minimum: 1). The number of messages accumulated on a log partition before messages are flushed to disk.log_flush_interval_ms (integer, Minimum: 0). The maximum time in ms that a message in any topic is kept in memory before flushed to disk. If not set, the value in log.flush.scheduler.interval.ms is used.log_index_interval_bytes (integer, Minimum: 0, Maximum: 104857600). The interval with which Kafka adds an entry to the offset index.log_index_size_max_bytes (integer, Minimum: 1048576, Maximum: 104857600). The maximum size in bytes of the offset index.log_local_retention_bytes (integer, Minimum: -2). The maximum size of local log segments that can grow for a partition before it gets eligible for deletion. If set to -2, the value of log.retention.bytes is used. The effective value should always be less than or equal to log.retention.bytes value.log_local_retention_ms (integer, Minimum: -2). The number of milliseconds to keep the local log segments before it gets eligible for deletion. If set to -2, the value of log.retention.ms is used. The effective value should always be less than or equal to log.retention.ms value.log_message_downconversion_enable (boolean). This configuration controls whether down-conversion of message formats is enabled to satisfy consume requests.log_message_timestamp_difference_max_ms (integer, Minimum: 0). The maximum difference allowed between the timestamp when a broker receives a message and the timestamp specified in the message.log_message_timestamp_type (string, Enum: CreateTime, LogAppendTime). Define whether the timestamp in the message is message create time or log append time.log_preallocate (boolean). Should pre allocate file when create new segment?.log_retention_bytes (integer, Minimum: -1). The maximum size of the log before deleting messages.log_retention_hours (integer, Minimum: -1, Maximum: 2147483647). The number of hours to keep a log file before deleting it.log_retention_ms (integer, Minimum: -1). The number of milliseconds to keep a log file before deleting it (in milliseconds), If not set, the value in log.retention.minutes is used. If set to -1, no time limit is applied.log_roll_jitter_ms (integer, Minimum: 0). The maximum jitter to subtract from logRollTimeMillis (in milliseconds). If not set, the value in log.roll.jitter.hours is used.log_roll_ms (integer, Minimum: 1). The maximum time before a new log segment is rolled out (in milliseconds).log_segment_bytes (integer, Minimum: 10485760, Maximum: 1073741824). The maximum size of a single log file.log_segment_delete_delay_ms (integer, Minimum: 0, Maximum: 3600000). The amount of time to wait before deleting a file from the filesystem.max_connections_per_ip (integer, Minimum: 256, Maximum: 2147483647). The maximum number of connections allowed from each ip address (defaults to 2147483647).max_incremental_fetch_session_cache_slots (integer, Minimum: 1000, Maximum: 10000). The maximum number of incremental fetch sessions that the broker will maintain.message_max_bytes (integer, Minimum: 0, Maximum: 100001200). The maximum size of message that the server can receive.min_insync_replicas (integer, Minimum: 1, Maximum: 7). When a producer sets acks to all (or -1), min.insync.replicas specifies the minimum number of replicas that must acknowledge a write for the write to be considered successful.num_partitions (integer, Minimum: 1, Maximum: 1000). Number of partitions for autocreated topics.offsets_retention_minutes (integer, Minimum: 1, Maximum: 2147483647). Log retention window in minutes for offsets topic.producer_purgatory_purge_interval_requests (integer, Minimum: 10, Maximum: 10000). The purge interval (in number of requests) of the producer request purgatory(defaults to 1000).replica_fetch_max_bytes (integer, Minimum: 1048576, Maximum: 104857600). The number of bytes of messages to attempt to fetch for each partition (defaults to 1048576). This is not an absolute maximum, if the first record batch in the first non-empty partition of the fetch is larger than this value, the record batch will still be returned to ensure that progress can be made.replica_fetch_response_max_bytes (integer, Minimum: 10485760, Maximum: 1048576000). Maximum bytes expected for the entire fetch response (defaults to 10485760). Records are fetched in batches, and if the first record batch in the first non-empty partition of the fetch is larger than this value, the record batch will still be returned to ensure that progress can be made. As such, this is not an absolute maximum.sasl_oauthbearer_expected_audience (string, Pattern: ^[^\\r\\n]*$, MaxLength: 128). The (optional) comma-delimited setting for the broker to use to verify that the JWT was issued for one of the expected audiences.sasl_oauthbearer_expected_issuer (string, Pattern: ^[^\\r\\n]*$, MaxLength: 128). Optional setting for the broker to use to verify that the JWT was created by the expected issuer.sasl_oauthbearer_jwks_endpoint_url (string, MaxLength: 2048). OIDC JWKS endpoint URL. By setting this the SASL SSL OAuth2/OIDC authentication is enabled. See also other options for SASL OAuth2/OIDC.sasl_oauthbearer_sub_claim_name (string, Pattern: ^[^\\r\\n]*$, MaxLength: 128). Name of the scope from which to extract the subject claim from the JWT. Defaults to sub.socket_request_max_bytes (integer, Minimum: 10485760, Maximum: 209715200). The maximum number of bytes in a socket request (defaults to 104857600).transaction_partition_verification_enable (boolean). Enable verification that checks that the partition has been added to the transaction before writing transactional records to the partition.transaction_remove_expired_transaction_cleanup_interval_ms (integer, Minimum: 600000, Maximum: 3600000). The interval at which to remove transactions that have expired due to transactional.id.expiration.ms passing (defaults to 3600000 (1 hour)).transaction_state_log_segment_bytes (integer, Minimum: 1048576, Maximum: 2147483647). The transaction topic segment bytes should be kept relatively small in order to facilitate faster log compaction and cache loads (defaults to 104857600 (100 mebibytes)).Appears on spec.userConfig.
Kafka authentication methods.
Optional
certificate (boolean). Enable certificate/SSL authentication.sasl (boolean). Enable SASL authentication.Appears on spec.userConfig.
Kafka Connect configuration values.
Optional
connector_client_config_override_policy (string, Enum: None, All). Defines what client configurations can be overridden by the connector. Default is None.consumer_auto_offset_reset (string, Enum: earliest, latest). What to do when there is no initial offset in Kafka or if the current offset does not exist any more on the server. Default is earliest.consumer_fetch_max_bytes (integer, Minimum: 1048576, Maximum: 104857600). Records are fetched in batches by the consumer, and if the first record batch in the first non-empty partition of the fetch is larger than this value, the record batch will still be returned to ensure that the consumer can make progress. As such, this is not a absolute maximum.consumer_isolation_level (string, Enum: read_uncommitted, read_committed). Transaction read isolation level. read_uncommitted is the default, but read_committed can be used if consume-exactly-once behavior is desired.consumer_max_partition_fetch_bytes (integer, Minimum: 1048576, Maximum: 104857600). Records are fetched in batches by the consumer.If the first record batch in the first non-empty partition of the fetch is larger than this limit, the batch will still be returned to ensure that the consumer can make progress.consumer_max_poll_interval_ms (integer, Minimum: 1, Maximum: 2147483647). The maximum delay in milliseconds between invocations of poll() when using consumer group management (defaults to 300000).consumer_max_poll_records (integer, Minimum: 1, Maximum: 10000). The maximum number of records returned in a single call to poll() (defaults to 500).offset_flush_interval_ms (integer, Minimum: 1, Maximum: 100000000). The interval at which to try committing offsets for tasks (defaults to 60000).offset_flush_timeout_ms (integer, Minimum: 1, Maximum: 2147483647). Maximum number of milliseconds to wait for records to flush and partition offset data to be committed to offset storage before cancelling the process and restoring the offset data to be committed in a future attempt (defaults to 5000).producer_batch_size (integer, Minimum: 0, Maximum: 5242880). This setting gives the upper bound of the batch size to be sent. If there are fewer than this many bytes accumulated for this partition, the producer will linger for the linger.ms time waiting for more records to show up. A batch size of zero will disable batching entirely (defaults to 16384).producer_buffer_memory (integer, Minimum: 5242880, Maximum: 134217728). The total bytes of memory the producer can use to buffer records waiting to be sent to the broker (defaults to 33554432).producer_compression_type (string, Enum: gzip, snappy, lz4, zstd, none). Specify the default compression type for producers. This configuration accepts the standard compression codecs (gzip, snappy, lz4, zstd). It additionally accepts none which is the default and equivalent to no compression.producer_linger_ms (integer, Minimum: 0, Maximum: 5000). This setting gives the upper bound on the delay for batching: once there is batch.size worth of records for a partition it will be sent immediately regardless of this setting, however if there are fewer than this many bytes accumulated for this partition the producer will linger for the specified time waiting for more records to show up. Defaults to 0.producer_max_request_size (integer, Minimum: 131072, Maximum: 67108864). This setting will limit the number of record batches the producer will send in a single request to avoid sending huge requests.scheduled_rebalance_max_delay_ms (integer, Minimum: 0, Maximum: 600000). The maximum delay that is scheduled in order to wait for the return of one or more departed workers before rebalancing and reassigning their connectors and tasks to the group. During this period the connectors and tasks of the departed workers remain unassigned. Defaults to 5 minutes.session_timeout_ms (integer, Minimum: 1, Maximum: 2147483647). The timeout in milliseconds used to detect failures when using Kafka\u2019s group management facilities (defaults to 10000).Appears on spec.userConfig.
Configure external secret providers in order to reference external secrets in connector configuration. Currently Hashicorp Vault (provider: vault, auth_method: token) and AWS Secrets Manager (provider: aws, auth_method: credentials) are supported. Secrets can be referenced in connector config with ${::}.
Required
name (string). Name of the secret provider. Used to reference secrets in connector config.Optional
aws (object). AWS config for Secret Provider. See below for nested schema.vault (object). Vault Config for Secret Provider. See below for nested schema.Appears on spec.userConfig.kafka_connect_secret_providers.
AWS config for Secret Provider.
Required
auth_method (string, Enum: credentials). Auth method of the vault secret provider.region (string, MaxLength: 64). Region used to lookup secrets with AWS SecretManager.Optional
access_key (string, MaxLength: 128). Access key used to authenticate with aws.secret_key (string, MaxLength: 128). Secret key used to authenticate with aws.Appears on spec.userConfig.kafka_connect_secret_providers.
Vault Config for Secret Provider.
Required
address (string, MinLength: 1, MaxLength: 65536). Address of the Vault server.auth_method (string, Enum: token). Auth method of the vault secret provider.Optional
engine_version (integer, Enum: 1, 2). KV Secrets Engine version of the Vault server instance.token (string, MaxLength: 256). Token used to authenticate with vault and auth method token.Appears on spec.userConfig.
Kafka REST configuration.
Optional
consumer_enable_auto_commit (boolean). If true the consumer's offset will be periodically committed to Kafka in the background.consumer_request_max_bytes (integer, Minimum: 0, Maximum: 671088640). Maximum number of bytes in unencoded message keys and values by a single request.consumer_request_timeout_ms (integer, Enum: 1000, 15000, 30000, Minimum: 1000, Maximum: 30000). The maximum total time to wait for messages for a request if the maximum number of messages has not yet been reached.name_strategy (string, Enum: topic_name, record_name, topic_record_name). Name strategy to use when selecting subject for storing schemas.name_strategy_validation (boolean). If true, validate that given schema is registered under expected subject name by the used name strategy when producing messages.producer_acks (string, Enum: all, -1, 0, 1). The number of acknowledgments the producer requires the leader to have received before considering a request complete. If set to all or -1, the leader will wait for the full set of in-sync replicas to acknowledge the record.producer_compression_type (string, Enum: gzip, snappy, lz4, zstd, none). Specify the default compression type for producers. This configuration accepts the standard compression codecs (gzip, snappy, lz4, zstd). It additionally accepts none which is the default and equivalent to no compression.producer_linger_ms (integer, Minimum: 0, Maximum: 5000). Wait for up to the given delay to allow batching records together.producer_max_request_size (integer, Minimum: 0, Maximum: 2147483647). The maximum size of a request in bytes. Note that Kafka broker can also cap the record batch size.simpleconsumer_pool_size_max (integer, Minimum: 10, Maximum: 250). Maximum number of SimpleConsumers that can be instantiated per broker.Appears on spec.userConfig.
Allow access to selected service ports from private networks.
Optional
kafka (boolean). Allow clients to connect to kafka with a DNS name that always resolves to the service's private IP addresses. Only available in certain network locations.kafka_connect (boolean). Allow clients to connect to kafka_connect with a DNS name that always resolves to the service's private IP addresses. Only available in certain network locations.kafka_rest (boolean). Allow clients to connect to kafka_rest with a DNS name that always resolves to the service's private IP addresses. Only available in certain network locations.prometheus (boolean). Allow clients to connect to prometheus with a DNS name that always resolves to the service's private IP addresses. Only available in certain network locations.schema_registry (boolean). Allow clients to connect to schema_registry with a DNS name that always resolves to the service's private IP addresses. Only available in certain network locations.Appears on spec.userConfig.
Allow access to selected service components through Privatelink.
Optional
jolokia (boolean). Enable jolokia.kafka (boolean). Enable kafka.kafka_connect (boolean). Enable kafka_connect.kafka_rest (boolean). Enable kafka_rest.prometheus (boolean). Enable prometheus.schema_registry (boolean). Enable schema_registry.Appears on spec.userConfig.
Allow access to selected service ports from the public Internet.
Optional
kafka (boolean). Allow clients to connect to kafka from the public internet for service nodes that are in a project VPC or another type of private network.kafka_connect (boolean). Allow clients to connect to kafka_connect from the public internet for service nodes that are in a project VPC or another type of private network.kafka_rest (boolean). Allow clients to connect to kafka_rest from the public internet for service nodes that are in a project VPC or another type of private network.prometheus (boolean). Allow clients to connect to prometheus from the public internet for service nodes that are in a project VPC or another type of private network.schema_registry (boolean). Allow clients to connect to schema_registry from the public internet for service nodes that are in a project VPC or another type of private network.Appears on spec.userConfig.
Schema Registry configuration.
Optional
leader_eligibility (boolean). If true, Karapace / Schema Registry on the service nodes can participate in leader election. It might be needed to disable this when the schemas topic is replicated to a secondary cluster and Karapace / Schema Registry there must not participate in leader election. Defaults to true.topic_name (string, MinLength: 1, MaxLength: 249). The durable single partition topic that acts as the durable log for the data. This topic must be compacted to avoid losing data due to retention policy. Please note that changing this configuration in an existing Schema Registry / Karapace setup leads to previous schemas being inaccessible, data encoded with them potentially unreadable and schema ID sequence put out of order. It's only possible to do the switch while Schema Registry / Karapace is disabled. Defaults to _schemas.Appears on spec.userConfig.
Tiered storage configuration.
Optional
enabled (boolean). Whether to enable the tiered storage functionality.local_cache (object). Deprecated. Local cache configuration. See below for nested schema.Appears on spec.userConfig.tiered_storage.
Deprecated. Local cache configuration.
Required
size (integer, Minimum: 1, Maximum: 107374182400). Deprecated. Local cache size in bytes.apiVersion: aiven.io/v1alpha1\nkind: KafkaACL\nmetadata:\n name: my-kafka-acl\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n project: my-aiven-project\n serviceName: my-kafka\n topic: my-topic\n username: my-user\n permission: admin\nInfo
To create this resource, a Secret containing Aiven token must be created first.
Apply the resource with:
kubectl apply -f example.yaml\nVerify the newly created KafkaACL:
kubectl get kafkaacls my-kafka-acl\nThe output is similar to the following:
Name Service Name Project Username Permission Topic \nmy-kafka-acl my-kafka my-aiven-project my-user admin my-topic \nKafkaACL is the Schema for the kafkaacls API.
Required
apiVersion (string). Value aiven.io/v1alpha1.kind (string). Value KafkaACL.metadata (object). Data that identifies the object, including a name string and optional namespace.spec (object). KafkaACLSpec defines the desired state of KafkaACL. See below for nested schema.Appears on KafkaACL.
KafkaACLSpec defines the desired state of KafkaACL.
Required
permission (string, Enum: admin, read, readwrite, write). Kafka permission to grant (admin, read, readwrite, write).project (string, Immutable, Pattern: ^[a-zA-Z0-9_-]+$, MaxLength: 63). Identifies the project this resource belongs to.serviceName (string, Immutable, Pattern: ^[a-z][-a-z0-9]+$, MaxLength: 63). Specifies the name of the service that this resource belongs to.topic (string). Topic name pattern for the ACL entry.username (string). Username pattern for the ACL entry.Optional
authSecretRef (object). Authentication reference to Aiven token in a secret. See below for nested schema.Appears on spec.
Authentication reference to Aiven token in a secret.
Required
key (string, MinLength: 1).name (string, MinLength: 1).apiVersion: aiven.io/v1alpha1\nkind: KafkaConnect\nmetadata:\n name: my-kafka-connect\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n project: my-aiven-project\n cloudName: google-europe-west1\n plan: business-4\n\n userConfig:\n kafka_connect:\n consumer_isolation_level: read_committed\n public_access:\n kafka_connect: true\nInfo
To create this resource, a Secret containing Aiven token must be created first.
Apply the resource with:
kubectl apply -f example.yaml\nVerify the newly created KafkaConnect:
kubectl get kafkaconnects my-kafka-connect\nThe output is similar to the following:
Name Project Region Plan State \nmy-kafka-connect my-aiven-project google-europe-west1 business-4 RUNNING \nKafkaConnect is the Schema for the kafkaconnects API.
Required
apiVersion (string). Value aiven.io/v1alpha1.kind (string). Value KafkaConnect.metadata (object). Data that identifies the object, including a name string and optional namespace.spec (object). KafkaConnectSpec defines the desired state of KafkaConnect. See below for nested schema.Appears on KafkaConnect.
KafkaConnectSpec defines the desired state of KafkaConnect.
Required
plan (string, MaxLength: 128). Subscription plan.project (string, Immutable, Pattern: ^[a-zA-Z0-9_-]+$, MaxLength: 63). Identifies the project this resource belongs to.Optional
authSecretRef (object). Authentication reference to Aiven token in a secret. See below for nested schema.cloudName (string, MaxLength: 256). Cloud the service runs in.maintenanceWindowDow (string, Enum: monday, tuesday, wednesday, thursday, friday, saturday, sunday). Day of week when maintenance operations should be performed. One monday, tuesday, wednesday, etc.maintenanceWindowTime (string, MaxLength: 8). Time of day when maintenance operations should be performed. UTC time in HH:mm:ss format.projectVPCRef (object). ProjectVPCRef reference to ProjectVPC resource to use its ID as ProjectVPCID automatically. See below for nested schema.projectVpcId (string, MaxLength: 36). Identifier of the VPC the service should be in, if any.serviceIntegrations (array of objects, Immutable, MaxItems: 1). Service integrations to specify when creating a service. Not applied after initial service creation. See below for nested schema.tags (object, AdditionalProperties: string). Tags are key-value pairs that allow you to categorize services.technicalEmails (array of objects, MaxItems: 10). Defines the email addresses that will receive alerts about upcoming maintenance updates or warnings about service instability. See below for nested schema.terminationProtection (boolean). Prevent service from being deleted. It is recommended to have this enabled for all services.userConfig (object). KafkaConnect specific user configuration options. See below for nested schema.Appears on spec.
Authentication reference to Aiven token in a secret.
Required
key (string, MinLength: 1).name (string, MinLength: 1).Appears on spec.
ProjectVPCRef reference to ProjectVPC resource to use its ID as ProjectVPCID automatically.
Required
name (string, MinLength: 1).Optional
namespace (string, MinLength: 1).Appears on spec.
Service integrations to specify when creating a service. Not applied after initial service creation.
Required
integrationType (string, Enum: read_replica).sourceServiceName (string, MinLength: 1, MaxLength: 64).Appears on spec.
Defines the email addresses that will receive alerts about upcoming maintenance updates or warnings about service instability.
Required
email (string). Email address.Appears on spec.
KafkaConnect specific user configuration options.
Optional
additional_backup_regions (array of strings, MaxItems: 1). Deprecated. Additional Cloud Regions for Backup Replication.ip_filter (array of objects, MaxItems: 1024). Allow incoming connections from CIDR address block, e.g. 10.20.0.0/16. See below for nested schema.kafka_connect (object). Kafka Connect configuration values. See below for nested schema.private_access (object). Allow access to selected service ports from private networks. See below for nested schema.privatelink_access (object). Allow access to selected service components through Privatelink. See below for nested schema.public_access (object). Allow access to selected service ports from the public Internet. See below for nested schema.secret_providers. See below for } (array of objects). Configure external secret providers in order to reference external secrets in connector configuration. Currently Hashicorp Vault (provider: vault, auth_method: token) and AWS Secrets Manager (provider: aws, auth_method: credentials) are supported. Secrets can be referenced in connector config with ${::nested schema. service_log (boolean). Store logs for the service so that they are available in the HTTP API and console.static_ips (boolean). Use static public IP addresses.Appears on spec.userConfig.
Allow incoming connections from CIDR address block, e.g. 10.20.0.0/16.
Required
network (string, MaxLength: 43). CIDR address block.Optional
description (string, MaxLength: 1024). Description for IP filter list entry.Appears on spec.userConfig.
Kafka Connect configuration values.
Optional
connector_client_config_override_policy (string, Enum: None, All). Defines what client configurations can be overridden by the connector. Default is None.consumer_auto_offset_reset (string, Enum: earliest, latest). What to do when there is no initial offset in Kafka or if the current offset does not exist any more on the server. Default is earliest.consumer_fetch_max_bytes (integer, Minimum: 1048576, Maximum: 104857600). Records are fetched in batches by the consumer, and if the first record batch in the first non-empty partition of the fetch is larger than this value, the record batch will still be returned to ensure that the consumer can make progress. As such, this is not a absolute maximum.consumer_isolation_level (string, Enum: read_uncommitted, read_committed). Transaction read isolation level. read_uncommitted is the default, but read_committed can be used if consume-exactly-once behavior is desired.consumer_max_partition_fetch_bytes (integer, Minimum: 1048576, Maximum: 104857600). Records are fetched in batches by the consumer.If the first record batch in the first non-empty partition of the fetch is larger than this limit, the batch will still be returned to ensure that the consumer can make progress.consumer_max_poll_interval_ms (integer, Minimum: 1, Maximum: 2147483647). The maximum delay in milliseconds between invocations of poll() when using consumer group management (defaults to 300000).consumer_max_poll_records (integer, Minimum: 1, Maximum: 10000). The maximum number of records returned in a single call to poll() (defaults to 500).offset_flush_interval_ms (integer, Minimum: 1, Maximum: 100000000). The interval at which to try committing offsets for tasks (defaults to 60000).offset_flush_timeout_ms (integer, Minimum: 1, Maximum: 2147483647). Maximum number of milliseconds to wait for records to flush and partition offset data to be committed to offset storage before cancelling the process and restoring the offset data to be committed in a future attempt (defaults to 5000).producer_batch_size (integer, Minimum: 0, Maximum: 5242880). This setting gives the upper bound of the batch size to be sent. If there are fewer than this many bytes accumulated for this partition, the producer will linger for the linger.ms time waiting for more records to show up. A batch size of zero will disable batching entirely (defaults to 16384).producer_buffer_memory (integer, Minimum: 5242880, Maximum: 134217728). The total bytes of memory the producer can use to buffer records waiting to be sent to the broker (defaults to 33554432).producer_compression_type (string, Enum: gzip, snappy, lz4, zstd, none). Specify the default compression type for producers. This configuration accepts the standard compression codecs (gzip, snappy, lz4, zstd). It additionally accepts none which is the default and equivalent to no compression.producer_linger_ms (integer, Minimum: 0, Maximum: 5000). This setting gives the upper bound on the delay for batching: once there is batch.size worth of records for a partition it will be sent immediately regardless of this setting, however if there are fewer than this many bytes accumulated for this partition the producer will linger for the specified time waiting for more records to show up. Defaults to 0.producer_max_request_size (integer, Minimum: 131072, Maximum: 67108864). This setting will limit the number of record batches the producer will send in a single request to avoid sending huge requests.scheduled_rebalance_max_delay_ms (integer, Minimum: 0, Maximum: 600000). The maximum delay that is scheduled in order to wait for the return of one or more departed workers before rebalancing and reassigning their connectors and tasks to the group. During this period the connectors and tasks of the departed workers remain unassigned. Defaults to 5 minutes.session_timeout_ms (integer, Minimum: 1, Maximum: 2147483647). The timeout in milliseconds used to detect failures when using Kafka\u2019s group management facilities (defaults to 10000).Appears on spec.userConfig.
Allow access to selected service ports from private networks.
Optional
kafka_connect (boolean). Allow clients to connect to kafka_connect with a DNS name that always resolves to the service's private IP addresses. Only available in certain network locations.prometheus (boolean). Allow clients to connect to prometheus with a DNS name that always resolves to the service's private IP addresses. Only available in certain network locations.Appears on spec.userConfig.
Allow access to selected service components through Privatelink.
Optional
jolokia (boolean). Enable jolokia.kafka_connect (boolean). Enable kafka_connect.prometheus (boolean). Enable prometheus.Appears on spec.userConfig.
Allow access to selected service ports from the public Internet.
Optional
kafka_connect (boolean). Allow clients to connect to kafka_connect from the public internet for service nodes that are in a project VPC or another type of private network.prometheus (boolean). Allow clients to connect to prometheus from the public internet for service nodes that are in a project VPC or another type of private network.Appears on spec.userConfig.
Configure external secret providers in order to reference external secrets in connector configuration. Currently Hashicorp Vault (provider: vault, auth_method: token) and AWS Secrets Manager (provider: aws, auth_method: credentials) are supported. Secrets can be referenced in connector config with ${::}.
Required
name (string). Name of the secret provider. Used to reference secrets in connector config.Optional
aws (object). AWS config for Secret Provider. See below for nested schema.vault (object). Vault Config for Secret Provider. See below for nested schema.Appears on spec.userConfig.secret_providers.
AWS config for Secret Provider.
Required
auth_method (string, Enum: credentials). Auth method of the vault secret provider.region (string, MaxLength: 64). Region used to lookup secrets with AWS SecretManager.Optional
access_key (string, MaxLength: 128). Access key used to authenticate with aws.secret_key (string, MaxLength: 128). Secret key used to authenticate with aws.Appears on spec.userConfig.secret_providers.
Vault Config for Secret Provider.
Required
address (string, MinLength: 1, MaxLength: 65536). Address of the Vault server.auth_method (string, Enum: token). Auth method of the vault secret provider.Optional
engine_version (integer, Enum: 1, 2). KV Secrets Engine version of the Vault server instance.token (string, MaxLength: 256). Token used to authenticate with vault and auth method token.KafkaConnector is the Schema for the kafkaconnectors API.
Required
apiVersion (string). Value aiven.io/v1alpha1.kind (string). Value KafkaConnector.metadata (object). Data that identifies the object, including a name string and optional namespace.spec (object). KafkaConnectorSpec defines the desired state of KafkaConnector. See below for nested schema.Appears on KafkaConnector.
KafkaConnectorSpec defines the desired state of KafkaConnector.
Required
connectorClass (string, MaxLength: 1024). The Java class of the connector.project (string, Immutable, Pattern: ^[a-zA-Z0-9_-]+$, MaxLength: 63). Identifies the project this resource belongs to.serviceName (string, Immutable, Pattern: ^[a-z][-a-z0-9]+$, MaxLength: 63). Specifies the name of the service that this resource belongs to.userConfig (object, AdditionalProperties: string). The connector specific configuration To build config values from secret the template function {{ fromSecret \"name\" \"key\" }} is provided when interpreting the keys.Optional
authSecretRef (object). Authentication reference to Aiven token in a secret. See below for nested schema.Appears on spec.
Authentication reference to Aiven token in a secret.
Required
key (string, MinLength: 1).name (string, MinLength: 1).apiVersion: aiven.io/v1alpha1\nkind: KafkaSchema\nmetadata:\n name: my-schema\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n project: my-aiven-project\n serviceName: my-kafka\n subjectName: mny-subject\n compatibilityLevel: BACKWARD\n schema: |\n {\n \"doc\": \"example_doc\",\n \"fields\": [{\n \"default\": 5,\n \"doc\": \"field_doc\",\n \"name\": \"field_name\",\n \"namespace\": \"field_namespace\",\n \"type\": \"int\"\n }],\n \"name\": \"example_name\",\n \"namespace\": \"example_namespace\",\n \"type\": \"record\"\n }\nInfo
To create this resource, a Secret containing Aiven token must be created first.
Apply the resource with:
kubectl apply -f example.yaml\nVerify the newly created KafkaSchema:
kubectl get kafkaschemas my-schema\nThe output is similar to the following:
Name Service Name Project Subject Compatibility Level Version \nmy-schema my-kafka my-aiven-project mny-subject BACKWARD <version> \nKafkaSchema is the Schema for the kafkaschemas API.
Required
apiVersion (string). Value aiven.io/v1alpha1.kind (string). Value KafkaSchema.metadata (object). Data that identifies the object, including a name string and optional namespace.spec (object). KafkaSchemaSpec defines the desired state of KafkaSchema. See below for nested schema.Appears on KafkaSchema.
KafkaSchemaSpec defines the desired state of KafkaSchema.
Required
project (string, Immutable, Pattern: ^[a-zA-Z0-9_-]+$, MaxLength: 63). Identifies the project this resource belongs to.schema (string). Kafka Schema configuration should be a valid Avro Schema JSON format.serviceName (string, Immutable, Pattern: ^[a-z][-a-z0-9]+$, MaxLength: 63). Specifies the name of the service that this resource belongs to.subjectName (string, MaxLength: 63). Kafka Schema Subject name.Optional
authSecretRef (object). Authentication reference to Aiven token in a secret. See below for nested schema.compatibilityLevel (string, Enum: BACKWARD, BACKWARD_TRANSITIVE, FORWARD, FORWARD_TRANSITIVE, FULL, FULL_TRANSITIVE, NONE). Kafka Schemas compatibility level.Appears on spec.
Authentication reference to Aiven token in a secret.
Required
key (string, MinLength: 1).name (string, MinLength: 1).apiVersion: aiven.io/v1alpha1\nkind: KafkaSchemaRegistryACL\nmetadata:\n name: my-kafka-schema-registry-acl\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n project: aiven-project-name\n serviceName: my-kafka\n resource: Subject:my-topic\n username: my-user\n permission: schema_registry_read\nInfo
To create this resource, a Secret containing Aiven token must be created first.
Apply the resource with:
kubectl apply -f example.yaml\nVerify the newly created KafkaSchemaRegistryACL:
kubectl get kafkaschemaregistryacls my-kafka-schema-registry-acl\nThe output is similar to the following:
Name Project Service Name Resource Username State \nmy-kafka-schema-registry-acl aiven-project-name my-kafka Subject:my-topic my-user RUNNING \nKafkaSchemaRegistryACL is the Schema for the kafkaschemaregistryacls API.
Required
apiVersion (string). Value aiven.io/v1alpha1.kind (string). Value KafkaSchemaRegistryACL.metadata (object). Data that identifies the object, including a name string and optional namespace.spec (object). KafkaSchemaRegistryACLSpec defines the desired state of KafkaSchemaRegistryACL. See below for nested schema.Appears on KafkaSchemaRegistryACL.
KafkaSchemaRegistryACLSpec defines the desired state of KafkaSchemaRegistryACL.
Required
permission (string, Enum: schema_registry_read, schema_registry_write, Immutable).project (string, Immutable, Pattern: ^[a-zA-Z0-9_-]+$, MaxLength: 63). Identifies the project this resource belongs to.resource (string, Immutable, MaxLength: 249). Resource name pattern for the Schema Registry ACL entry.serviceName (string, Immutable, Pattern: ^[a-z][-a-z0-9]+$, MaxLength: 63). Specifies the name of the service that this resource belongs to.username (string, Immutable, MaxLength: 64). Username pattern for the ACL entry.Optional
authSecretRef (object). Authentication reference to Aiven token in a secret. See below for nested schema.Appears on spec.
Authentication reference to Aiven token in a secret.
Required
key (string, MinLength: 1).name (string, MinLength: 1).apiVersion: aiven.io/v1alpha1\nkind: KafkaTopic\nmetadata:\n name: kafka-topic\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n project: my-aiven-project\n serviceName: my-kafka\n topicName: my-kafka-topic\n\n replication: 2\n partitions: 1\n\n config:\n min_cleanable_dirty_ratio: 0.2\nInfo
To create this resource, a Secret containing Aiven token must be created first.
Apply the resource with:
kubectl apply -f example.yaml\nVerify the newly created KafkaTopic:
kubectl get kafkatopics kafka-topic\nThe output is similar to the following:
Name Service Name Project Partitions Replication \nkafka-topic my-kafka my-aiven-project 1 2 \nKafkaTopic is the Schema for the kafkatopics API.
Required
apiVersion (string). Value aiven.io/v1alpha1.kind (string). Value KafkaTopic.metadata (object). Data that identifies the object, including a name string and optional namespace.spec (object). KafkaTopicSpec defines the desired state of KafkaTopic. See below for nested schema.Appears on KafkaTopic.
KafkaTopicSpec defines the desired state of KafkaTopic.
Required
partitions (integer, Minimum: 1, Maximum: 1000000). Number of partitions to create in the topic.project (string, Immutable, Pattern: ^[a-zA-Z0-9_-]+$, MaxLength: 63). Identifies the project this resource belongs to.replication (integer, Minimum: 2). Replication factor for the topic.serviceName (string, Immutable, Pattern: ^[a-z][-a-z0-9]+$, MaxLength: 63). Specifies the name of the service that this resource belongs to.Optional
authSecretRef (object). Authentication reference to Aiven token in a secret. See below for nested schema.config (object). Kafka topic configuration. See below for nested schema.tags (array of objects). Kafka topic tags. See below for nested schema.termination_protection (boolean). It is a Kubernetes side deletion protections, which prevents the kafka topic from being deleted by Kubernetes. It is recommended to enable this for any production databases containing critical data.topicName (string, Immutable, MinLength: 1, MaxLength: 249). Topic name. If provided, is used instead of metadata.name. This field supports additional characters, has a longer length, and will replace metadata.name in future releases.Appears on spec.
Authentication reference to Aiven token in a secret.
Required
key (string, MinLength: 1).name (string, MinLength: 1).Appears on spec.
Kafka topic configuration.
Optional
cleanup_policy (string). cleanup.policy value.compression_type (string). compression.type value.delete_retention_ms (integer). delete.retention.ms value.file_delete_delay_ms (integer). file.delete.delay.ms value.flush_messages (integer). flush.messages value.flush_ms (integer). flush.ms value.index_interval_bytes (integer). index.interval.bytes value.local_retention_bytes (integer). local.retention.bytes value.local_retention_ms (integer). local.retention.ms value.max_compaction_lag_ms (integer). max.compaction.lag.ms value.max_message_bytes (integer). max.message.bytes value.message_downconversion_enable (boolean). message.downconversion.enable value.message_format_version (string). message.format.version value.message_timestamp_difference_max_ms (integer). message.timestamp.difference.max.ms value.message_timestamp_type (string). message.timestamp.type value.min_cleanable_dirty_ratio (number). min.cleanable.dirty.ratio value.min_compaction_lag_ms (integer). min.compaction.lag.ms value.min_insync_replicas (integer). min.insync.replicas value.preallocate (boolean). preallocate value.remote_storage_enable (boolean). remote_storage_enable.retention_bytes (integer). retention.bytes value.retention_ms (integer). retention.ms value.segment_bytes (integer). segment.bytes value.segment_index_bytes (integer). segment.index.bytes value.segment_jitter_ms (integer). segment.jitter.ms value.segment_ms (integer). segment.ms value.Appears on spec.
Kafka topic tags.
Required
key (string, Pattern: ^[a-zA-Z0-9_-]+$, MinLength: 1, MaxLength: 64).Optional
value (string, Pattern: ^[a-zA-Z0-9_-]+$, MaxLength: 256).apiVersion: aiven.io/v1alpha1\nkind: MySQL\nmetadata:\n name: my-mysql\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n connInfoSecretTarget:\n name: mysql-secret\n prefix: MY_SECRET_PREFIX_\n annotations:\n foo: bar\n labels:\n baz: egg\n\n project: my-aiven-project\n cloudName: google-europe-west1\n plan: business-4\n\n maintenanceWindowDow: sunday\n maintenanceWindowTime: 11:00:00\n\n userConfig:\n backup_hour: 17\n backup_minute: 11\n ip_filter:\n - network: 0.0.0.0\n description: whatever\n - network: 10.20.0.0/16\nInfo
To create this resource, a Secret containing Aiven token must be created first.
Apply the resource with:
kubectl apply -f example.yaml\nVerify the newly created MySQL:
kubectl get mysqls my-mysql\nThe output is similar to the following:
Name Project Region Plan State \nmy-mysql my-aiven-project google-europe-west1 business-4 RUNNING \nTo view the details of the Secret, use the following command:
kubectl describe secret mysql-secret\nYou can use the jq to quickly decode the Secret:
kubectl get secret mysql-secret -o json | jq '.data | map_values(@base64d)'\nThe output is similar to the following:
{\n \"MYSQL_HOST\": \"<secret>\",\n \"MYSQL_PORT\": \"<secret>\",\n \"MYSQL_DATABASE\": \"<secret>\",\n \"MYSQL_USER\": \"<secret>\",\n \"MYSQL_PASSWORD\": \"<secret>\",\n \"MYSQL_SSL_MODE\": \"<secret>\",\n \"MYSQL_URI\": \"<secret>\",\n \"MYSQL_REPLICA_URI\": \"<secret>\",\n \"MYSQL_CA_CERT\": \"<secret>\",\n}\nMySQL is the Schema for the mysqls API.
Exposes secret keys
MYSQL_HOST, MYSQL_PORT, MYSQL_DATABASE, MYSQL_USER, MYSQL_PASSWORD, MYSQL_SSL_MODE, MYSQL_URI, MYSQL_REPLICA_URI, MYSQL_CA_CERT.
Required
apiVersion (string). Value aiven.io/v1alpha1.kind (string). Value MySQL.metadata (object). Data that identifies the object, including a name string and optional namespace.spec (object). MySQLSpec defines the desired state of MySQL. See below for nested schema.Appears on MySQL.
MySQLSpec defines the desired state of MySQL.
Required
plan (string, MaxLength: 128). Subscription plan.project (string, Immutable, Pattern: ^[a-zA-Z0-9_-]+$, MaxLength: 63). Identifies the project this resource belongs to.Optional
authSecretRef (object). Authentication reference to Aiven token in a secret. See below for nested schema.cloudName (string, MaxLength: 256). Cloud the service runs in.connInfoSecretTarget (object). Secret configuration. See below for nested schema.connInfoSecretTargetDisabled (boolean, Immutable). When true, the secret containing connection information will not be created, defaults to false. This field cannot be changed after resource creation.disk_space (string, Pattern: (?i)^[1-9][0-9]*(GiB|G)?$). The disk space of the service, possible values depend on the service type, the cloud provider and the project. Reducing will result in the service re-balancing. The removal of this field does not change the value.maintenanceWindowDow (string, Enum: monday, tuesday, wednesday, thursday, friday, saturday, sunday). Day of week when maintenance operations should be performed. One monday, tuesday, wednesday, etc.maintenanceWindowTime (string, MaxLength: 8). Time of day when maintenance operations should be performed. UTC time in HH:mm:ss format.projectVPCRef (object). ProjectVPCRef reference to ProjectVPC resource to use its ID as ProjectVPCID automatically. See below for nested schema.projectVpcId (string, MaxLength: 36). Identifier of the VPC the service should be in, if any.serviceIntegrations (array of objects, Immutable, MaxItems: 1). Service integrations to specify when creating a service. Not applied after initial service creation. See below for nested schema.tags (object, AdditionalProperties: string). Tags are key-value pairs that allow you to categorize services.technicalEmails (array of objects, MaxItems: 10). Defines the email addresses that will receive alerts about upcoming maintenance updates or warnings about service instability. See below for nested schema.terminationProtection (boolean). Prevent service from being deleted. It is recommended to have this enabled for all services.userConfig (object). MySQL specific user configuration options. See below for nested schema.Appears on spec.
Authentication reference to Aiven token in a secret.
Required
key (string, MinLength: 1).name (string, MinLength: 1).Appears on spec.
Secret configuration.
Required
name (string, Immutable). Name of the secret resource to be created. By default, it is equal to the resource name.Optional
annotations (object, AdditionalProperties: string). Annotations added to the secret.labels (object, AdditionalProperties: string). Labels added to the secret.prefix (string). Prefix for the secret's keys. Added \"as is\" without any transformations. By default, is equal to the kind name in uppercase + underscore, e.g. KAFKA_, REDIS_, etc.Appears on spec.
ProjectVPCRef reference to ProjectVPC resource to use its ID as ProjectVPCID automatically.
Required
name (string, MinLength: 1).Optional
namespace (string, MinLength: 1).Appears on spec.
Service integrations to specify when creating a service. Not applied after initial service creation.
Required
integrationType (string, Enum: read_replica).sourceServiceName (string, MinLength: 1, MaxLength: 64).Appears on spec.
Defines the email addresses that will receive alerts about upcoming maintenance updates or warnings about service instability.
Required
email (string). Email address.Appears on spec.
MySQL specific user configuration options.
Optional
additional_backup_regions (array of strings, MaxItems: 1). Additional Cloud Regions for Backup Replication.admin_password (string, Immutable, Pattern: ^[a-zA-Z0-9-_]+$, MinLength: 8, MaxLength: 256). Custom password for admin user. Defaults to random string. This must be set only when a new service is being created.admin_username (string, Immutable, Pattern: ^[_A-Za-z0-9][-._A-Za-z0-9]{0,63}$, MaxLength: 64). Custom username for admin user. This must be set only when a new service is being created.backup_hour (integer, Minimum: 0, Maximum: 23). The hour of day (in UTC) when backup for the service is started. New backup is only started if previous backup has already completed.backup_minute (integer, Minimum: 0, Maximum: 59). The minute of an hour when backup for the service is started. New backup is only started if previous backup has already completed.binlog_retention_period (integer, Minimum: 600, Maximum: 86400). The minimum amount of time in seconds to keep binlog entries before deletion. This may be extended for services that require binlog entries for longer than the default for example if using the MySQL Debezium Kafka connector.ip_filter (array of objects, MaxItems: 1024). Allow incoming connections from CIDR address block, e.g. 10.20.0.0/16. See below for nested schema.migration (object). Migrate data from existing server. See below for nested schema.mysql (object). mysql.conf configuration values. See below for nested schema.mysql_version (string, Enum: 8). MySQL major version.private_access (object). Allow access to selected service ports from private networks. See below for nested schema.privatelink_access (object). Allow access to selected service components through Privatelink. See below for nested schema.project_to_fork_from (string, Immutable, Pattern: ^[a-z][-a-z0-9]{0,63}$|^$, MaxLength: 63). Name of another project to fork a service from. This has effect only when a new service is being created.public_access (object). Allow access to selected service ports from the public Internet. See below for nested schema.recovery_target_time (string, Immutable, MaxLength: 32). Recovery target time when forking a service. This has effect only when a new service is being created.service_log (boolean). Store logs for the service so that they are available in the HTTP API and console.service_to_fork_from (string, Immutable, Pattern: ^[a-z][-a-z0-9]{0,63}$|^$, MaxLength: 64). Name of another service to fork from. This has effect only when a new service is being created.static_ips (boolean). Use static public IP addresses.Appears on spec.userConfig.
Allow incoming connections from CIDR address block, e.g. 10.20.0.0/16.
Required
network (string, MaxLength: 43). CIDR address block.Optional
description (string, MaxLength: 1024). Description for IP filter list entry.Appears on spec.userConfig.
Migrate data from existing server.
Required
host (string, MaxLength: 255). Hostname or IP address of the server where to migrate data from.port (integer, Minimum: 1, Maximum: 65535). Port number of the server where to migrate data from.Optional
dbname (string, MaxLength: 63). Database name for bootstrapping the initial connection.ignore_dbs (string, MaxLength: 2048). Comma-separated list of databases, which should be ignored during migration (supported by MySQL and PostgreSQL only at the moment).method (string, Enum: dump, replication). The migration method to be used (currently supported only by Redis, Dragonfly, MySQL and PostgreSQL service types).password (string, MaxLength: 256). Password for authentication with the server where to migrate data from.ssl (boolean). The server where to migrate data from is secured with SSL.username (string, MaxLength: 256). User name for authentication with the server where to migrate data from.Appears on spec.userConfig.
mysql.conf configuration values.
Optional
connect_timeout (integer, Minimum: 2, Maximum: 3600). The number of seconds that the mysqld server waits for a connect packet before responding with Bad handshake.default_time_zone (string, Pattern: ^([-+][\\d:]*|[\\w/]*)$, MinLength: 2, MaxLength: 100). Default server time zone as an offset from UTC (from -12:00 to +12:00), a time zone name, or SYSTEM to use the MySQL server default.group_concat_max_len (integer, Minimum: 4). The maximum permitted result length in bytes for the GROUP_CONCAT() function.information_schema_stats_expiry (integer, Minimum: 900, Maximum: 31536000). The time, in seconds, before cached statistics expire.innodb_change_buffer_max_size (integer, Minimum: 0, Maximum: 50). Maximum size for the InnoDB change buffer, as a percentage of the total size of the buffer pool. Default is 25.innodb_flush_neighbors (integer, Minimum: 0, Maximum: 2). Specifies whether flushing a page from the InnoDB buffer pool also flushes other dirty pages in the same extent (default is 1): 0 - dirty pages in the same extent are not flushed, 1 - flush contiguous dirty pages in the same extent, 2 - flush dirty pages in the same extent.innodb_ft_min_token_size (integer, Minimum: 0, Maximum: 16). Minimum length of words that are stored in an InnoDB FULLTEXT index. Changing this parameter will lead to a restart of the MySQL service.innodb_ft_server_stopword_table (string, Pattern: ^.+/.+$, MaxLength: 1024). This option is used to specify your own InnoDB FULLTEXT index stopword list for all InnoDB tables.innodb_lock_wait_timeout (integer, Minimum: 1, Maximum: 3600). The length of time in seconds an InnoDB transaction waits for a row lock before giving up. Default is 120.innodb_log_buffer_size (integer, Minimum: 1048576, Maximum: 4294967295). The size in bytes of the buffer that InnoDB uses to write to the log files on disk.innodb_online_alter_log_max_size (integer, Minimum: 65536, Maximum: 1099511627776). The upper limit in bytes on the size of the temporary log files used during online DDL operations for InnoDB tables.innodb_print_all_deadlocks (boolean). When enabled, information about all deadlocks in InnoDB user transactions is recorded in the error log. Disabled by default.innodb_read_io_threads (integer, Minimum: 1, Maximum: 64). The number of I/O threads for read operations in InnoDB. Default is 4. Changing this parameter will lead to a restart of the MySQL service.innodb_rollback_on_timeout (boolean). When enabled a transaction timeout causes InnoDB to abort and roll back the entire transaction. Changing this parameter will lead to a restart of the MySQL service.innodb_thread_concurrency (integer, Minimum: 0, Maximum: 1000). Defines the maximum number of threads permitted inside of InnoDB. Default is 0 (infinite concurrency - no limit).innodb_write_io_threads (integer, Minimum: 1, Maximum: 64). The number of I/O threads for write operations in InnoDB. Default is 4. Changing this parameter will lead to a restart of the MySQL service.interactive_timeout (integer, Minimum: 30, Maximum: 604800). The number of seconds the server waits for activity on an interactive connection before closing it.internal_tmp_mem_storage_engine (string, Enum: TempTable, MEMORY). The storage engine for in-memory internal temporary tables.long_query_time (number, Minimum: 0, Maximum: 3600). The slow_query_logs work as SQL statements that take more than long_query_time seconds to execute. Default is 10s.max_allowed_packet (integer, Minimum: 102400, Maximum: 1073741824). Size of the largest message in bytes that can be received by the server. Default is 67108864 (64M).max_heap_table_size (integer, Minimum: 1048576, Maximum: 1073741824). Limits the size of internal in-memory tables. Also set tmp_table_size. Default is 16777216 (16M).net_buffer_length (integer, Minimum: 1024, Maximum: 1048576). Start sizes of connection buffer and result buffer. Default is 16384 (16K). Changing this parameter will lead to a restart of the MySQL service.net_read_timeout (integer, Minimum: 1, Maximum: 3600). The number of seconds to wait for more data from a connection before aborting the read.net_write_timeout (integer, Minimum: 1, Maximum: 3600). The number of seconds to wait for a block to be written to a connection before aborting the write.slow_query_log (boolean). Slow query log enables capturing of slow queries. Setting slow_query_log to false also truncates the mysql.slow_log table. Default is off.sort_buffer_size (integer, Minimum: 32768, Maximum: 1073741824). Sort buffer size in bytes for ORDER BY optimization. Default is 262144 (256K).sql_mode (string, Pattern: ^[A-Z_]*(,[A-Z_]+)*$, MaxLength: 1024). Global SQL mode. Set to empty to use MySQL server defaults. When creating a new service and not setting this field Aiven default SQL mode (strict, SQL standard compliant) will be assigned.sql_require_primary_key (boolean). Require primary key to be defined for new tables or old tables modified with ALTER TABLE and fail if missing. It is recommended to always have primary keys because various functionality may break if any large table is missing them.tmp_table_size (integer, Minimum: 1048576, Maximum: 1073741824). Limits the size of internal in-memory tables. Also set max_heap_table_size. Default is 16777216 (16M).wait_timeout (integer, Minimum: 1, Maximum: 2147483). The number of seconds the server waits for activity on a noninteractive connection before closing it.Appears on spec.userConfig.
Allow access to selected service ports from private networks.
Optional
mysql (boolean). Allow clients to connect to mysql with a DNS name that always resolves to the service's private IP addresses. Only available in certain network locations.mysqlx (boolean). Allow clients to connect to mysqlx with a DNS name that always resolves to the service's private IP addresses. Only available in certain network locations.prometheus (boolean). Allow clients to connect to prometheus with a DNS name that always resolves to the service's private IP addresses. Only available in certain network locations.Appears on spec.userConfig.
Allow access to selected service components through Privatelink.
Optional
mysql (boolean). Enable mysql.mysqlx (boolean). Enable mysqlx.prometheus (boolean). Enable prometheus.Appears on spec.userConfig.
Allow access to selected service ports from the public Internet.
Optional
mysql (boolean). Allow clients to connect to mysql from the public internet for service nodes that are in a project VPC or another type of private network.mysqlx (boolean). Allow clients to connect to mysqlx from the public internet for service nodes that are in a project VPC or another type of private network.prometheus (boolean). Allow clients to connect to prometheus from the public internet for service nodes that are in a project VPC or another type of private network.apiVersion: aiven.io/v1alpha1\nkind: OpenSearch\nmetadata:\n name: my-os\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n connInfoSecretTarget:\n name: os-secret\n prefix: MY_SECRET_PREFIX_\n annotations:\n foo: bar\n labels:\n baz: egg\n\n project: my-aiven-project\n cloudName: google-europe-west1\n plan: startup-4\n disk_space: 80GiB\n\n maintenanceWindowDow: friday\n maintenanceWindowTime: 23:00:00\nInfo
To create this resource, a Secret containing Aiven token must be created first.
Apply the resource with:
kubectl apply -f example.yaml\nVerify the newly created OpenSearch:
kubectl get opensearches my-os\nThe output is similar to the following:
Name Project Region Plan State \nmy-os my-aiven-project google-europe-west1 startup-4 RUNNING \nTo view the details of the Secret, use the following command:
kubectl describe secret os-secret\nYou can use the jq to quickly decode the Secret:
kubectl get secret os-secret -o json | jq '.data | map_values(@base64d)'\nThe output is similar to the following:
{\n \"OPENSEARCH_HOST\": \"<secret>\",\n \"OPENSEARCH_PORT\": \"<secret>\",\n \"OPENSEARCH_USER\": \"<secret>\",\n \"OPENSEARCH_PASSWORD\": \"<secret>\",\n}\nOpenSearch is the Schema for the opensearches API.
Exposes secret keys
OPENSEARCH_HOST, OPENSEARCH_PORT, OPENSEARCH_USER, OPENSEARCH_PASSWORD.
Required
apiVersion (string). Value aiven.io/v1alpha1.kind (string). Value OpenSearch.metadata (object). Data that identifies the object, including a name string and optional namespace.spec (object). OpenSearchSpec defines the desired state of OpenSearch. See below for nested schema.Appears on OpenSearch.
OpenSearchSpec defines the desired state of OpenSearch.
Required
plan (string, MaxLength: 128). Subscription plan.project (string, Immutable, Pattern: ^[a-zA-Z0-9_-]+$, MaxLength: 63). Identifies the project this resource belongs to.Optional
authSecretRef (object). Authentication reference to Aiven token in a secret. See below for nested schema.cloudName (string, MaxLength: 256). Cloud the service runs in.connInfoSecretTarget (object). Secret configuration. See below for nested schema.connInfoSecretTargetDisabled (boolean, Immutable). When true, the secret containing connection information will not be created, defaults to false. This field cannot be changed after resource creation.disk_space (string, Pattern: (?i)^[1-9][0-9]*(GiB|G)?$). The disk space of the service, possible values depend on the service type, the cloud provider and the project. Reducing will result in the service re-balancing. The removal of this field does not change the value.maintenanceWindowDow (string, Enum: monday, tuesday, wednesday, thursday, friday, saturday, sunday). Day of week when maintenance operations should be performed. One monday, tuesday, wednesday, etc.maintenanceWindowTime (string, MaxLength: 8). Time of day when maintenance operations should be performed. UTC time in HH:mm:ss format.projectVPCRef (object). ProjectVPCRef reference to ProjectVPC resource to use its ID as ProjectVPCID automatically. See below for nested schema.projectVpcId (string, MaxLength: 36). Identifier of the VPC the service should be in, if any.serviceIntegrations (array of objects, Immutable, MaxItems: 1). Service integrations to specify when creating a service. Not applied after initial service creation. See below for nested schema.tags (object, AdditionalProperties: string). Tags are key-value pairs that allow you to categorize services.technicalEmails (array of objects, MaxItems: 10). Defines the email addresses that will receive alerts about upcoming maintenance updates or warnings about service instability. See below for nested schema.terminationProtection (boolean). Prevent service from being deleted. It is recommended to have this enabled for all services.userConfig (object). OpenSearch specific user configuration options. See below for nested schema.Appears on spec.
Authentication reference to Aiven token in a secret.
Required
key (string, MinLength: 1).name (string, MinLength: 1).Appears on spec.
Secret configuration.
Required
name (string, Immutable). Name of the secret resource to be created. By default, it is equal to the resource name.Optional
annotations (object, AdditionalProperties: string). Annotations added to the secret.labels (object, AdditionalProperties: string). Labels added to the secret.prefix (string). Prefix for the secret's keys. Added \"as is\" without any transformations. By default, is equal to the kind name in uppercase + underscore, e.g. KAFKA_, REDIS_, etc.Appears on spec.
ProjectVPCRef reference to ProjectVPC resource to use its ID as ProjectVPCID automatically.
Required
name (string, MinLength: 1).Optional
namespace (string, MinLength: 1).Appears on spec.
Service integrations to specify when creating a service. Not applied after initial service creation.
Required
integrationType (string, Enum: read_replica).sourceServiceName (string, MinLength: 1, MaxLength: 64).Appears on spec.
Defines the email addresses that will receive alerts about upcoming maintenance updates or warnings about service instability.
Required
email (string). Email address.Appears on spec.
OpenSearch specific user configuration options.
Optional
additional_backup_regions (array of strings, MaxItems: 1). Additional Cloud Regions for Backup Replication.custom_domain (string, MaxLength: 255). Serve the web frontend using a custom CNAME pointing to the Aiven DNS name.disable_replication_factor_adjustment (boolean). DEPRECATED: Disable automatic replication factor adjustment for multi-node services. By default, Aiven ensures all indexes are replicated at least to two nodes. Note: Due to potential data loss in case of losing a service node, this setting can no longer be activated.index_patterns (array of objects, MaxItems: 512). Index patterns. See below for nested schema.index_template (object). Template settings for all new indexes. See below for nested schema.ip_filter (array of objects, MaxItems: 1024). Allow incoming connections from CIDR address block, e.g. 10.20.0.0/16. See below for nested schema.keep_index_refresh_interval (boolean). Aiven automation resets index.refresh_interval to default value for every index to be sure that indices are always visible to search. If it doesn't fit your case, you can disable this by setting up this flag to true.max_index_count (integer, Minimum: 0). DEPRECATED: use index_patterns instead.openid (object). OpenSearch OpenID Connect Configuration. See below for nested schema.opensearch (object). OpenSearch settings. See below for nested schema.opensearch_dashboards (object). OpenSearch Dashboards settings. See below for nested schema.opensearch_version (string, Enum: 1, 2). OpenSearch major version.private_access (object). Allow access to selected service ports from private networks. See below for nested schema.privatelink_access (object). Allow access to selected service components through Privatelink. See below for nested schema.project_to_fork_from (string, Immutable, Pattern: ^[a-z][-a-z0-9]{0,63}$|^$, MaxLength: 63). Name of another project to fork a service from. This has effect only when a new service is being created.public_access (object). Allow access to selected service ports from the public Internet. See below for nested schema.recovery_basebackup_name (string, Pattern: ^[a-zA-Z0-9-_:.]+$, MaxLength: 128). Name of the basebackup to restore in forked service.saml (object). OpenSearch SAML configuration. See below for nested schema.service_log (boolean). Store logs for the service so that they are available in the HTTP API and console.service_to_fork_from (string, Immutable, Pattern: ^[a-z][-a-z0-9]{0,63}$|^$, MaxLength: 64). Name of another service to fork from. This has effect only when a new service is being created.static_ips (boolean). Use static public IP addresses.Appears on spec.userConfig.
Index patterns.
Required
max_index_count (integer, Minimum: 0). Maximum number of indexes to keep.pattern (string, Pattern: ^[A-Za-z0-9-_.*?]+$, MaxLength: 1024). fnmatch pattern.Optional
sorting_algorithm (string, Enum: alphabetical, creation_date). Deletion sorting algorithm.Appears on spec.userConfig.
Template settings for all new indexes.
Optional
mapping_nested_objects_limit (integer, Minimum: 0, Maximum: 100000). The maximum number of nested JSON objects that a single document can contain across all nested types. This limit helps to prevent out of memory errors when a document contains too many nested objects. Default is 10000.number_of_replicas (integer, Minimum: 0, Maximum: 29). The number of replicas each primary shard has.number_of_shards (integer, Minimum: 1, Maximum: 1024). The number of primary shards that an index should have.Appears on spec.userConfig.
Allow incoming connections from CIDR address block, e.g. 10.20.0.0/16.
Required
network (string, MaxLength: 43). CIDR address block.Optional
description (string, MaxLength: 1024). Description for IP filter list entry.Appears on spec.userConfig.
OpenSearch OpenID Connect Configuration.
Required
client_id (string, Pattern: ^[^\\r\\n]*$, MinLength: 1, MaxLength: 1024). The ID of the OpenID Connect client configured in your IdP. Required.client_secret (string, Pattern: ^[^\\r\\n]*$, MinLength: 1, MaxLength: 1024). The client secret of the OpenID Connect client configured in your IdP. Required.connect_url (string, MaxLength: 2048). The URL of your IdP where the Security plugin can find the OpenID Connect metadata/configuration settings.enabled (boolean). Enables or disables OpenID Connect authentication for OpenSearch. When enabled, users can authenticate using OpenID Connect with an Identity Provider.Optional
header (string, Pattern: ^[^\\r\\n]*$, MinLength: 1, MaxLength: 1024). HTTP header name of the JWT token. Optional. Default is Authorization.jwt_header (string, Pattern: ^[^\\r\\n]*$, MinLength: 1, MaxLength: 1024). The HTTP header that stores the token. Typically the Authorization header with the Bearer schema: Authorization: Bearer . Optional. Default is Authorization. jwt_url_parameter (string, Pattern: ^[^\\r\\n]*$, MinLength: 1, MaxLength: 1024). If the token is not transmitted in the HTTP header, but as an URL parameter, define the name of the parameter here. Optional.refresh_rate_limit_count (integer, Minimum: 10). The maximum number of unknown key IDs in the time frame. Default is 10. Optional.refresh_rate_limit_time_window_ms (integer, Minimum: 10000). The time frame to use when checking the maximum number of unknown key IDs, in milliseconds. Optional.Default is 10000 (10 seconds).roles_key (string, Pattern: ^[^\\r\\n]*$, MinLength: 1, MaxLength: 1024). The key in the JSON payload that stores the user\u2019s roles. The value of this key must be a comma-separated list of roles. Required only if you want to use roles in the JWT.scope (string, Pattern: ^[^\\r\\n]*$, MinLength: 1, MaxLength: 1024). The scope of the identity token issued by the IdP. Optional. Default is openid profile email address phone.subject_key (string, Pattern: ^[^\\r\\n]*$, MinLength: 1, MaxLength: 1024). The key in the JSON payload that stores the user\u2019s name. If not defined, the subject registered claim is used. Most IdP providers use the preferred_username claim. Optional.Appears on spec.userConfig.
OpenSearch settings.
Optional
action_auto_create_index_enabled (boolean). Explicitly allow or block automatic creation of indices. Defaults to true.action_destructive_requires_name (boolean). Require explicit index names when deleting.auth_failure_listeners (object). Opensearch Security Plugin Settings. See below for nested schema.cluster_max_shards_per_node (integer, Minimum: 100, Maximum: 10000). Controls the number of shards allowed in the cluster per data node.cluster_routing_allocation_node_concurrent_recoveries (integer, Minimum: 2, Maximum: 16). How many concurrent incoming/outgoing shard recoveries (normally replicas) are allowed to happen on a node. Defaults to 2.email_sender_name (string, Pattern: ^[a-zA-Z0-9-_]+$, MaxLength: 40). Sender name placeholder to be used in Opensearch Dashboards and Opensearch keystore.email_sender_password (string, Pattern: ^[^\\x00-\\x1F]+$, MaxLength: 1024). Sender password for Opensearch alerts to authenticate with SMTP server.email_sender_username (string, Pattern: ^[^\\x00-\\x1F]+$, MaxLength: 320). Sender username for Opensearch alerts.enable_security_audit (boolean). Enable/Disable security audit.http_max_content_length (integer, Minimum: 1, Maximum: 2147483647). Maximum content length for HTTP requests to the OpenSearch HTTP API, in bytes.http_max_header_size (integer, Minimum: 1024, Maximum: 262144). The max size of allowed headers, in bytes.http_max_initial_line_length (integer, Minimum: 1024, Maximum: 65536). The max length of an HTTP URL, in bytes.indices_fielddata_cache_size (integer, Minimum: 3, Maximum: 100). Relative amount. Maximum amount of heap memory used for field data cache. This is an expert setting; decreasing the value too much will increase overhead of loading field data; too much memory used for field data cache will decrease amount of heap available for other operations.indices_memory_index_buffer_size (integer, Minimum: 3, Maximum: 40). Percentage value. Default is 10%. Total amount of heap used for indexing buffer, before writing segments to disk. This is an expert setting. Too low value will slow down indexing; too high value will increase indexing performance but causes performance issues for query performance.indices_memory_max_index_buffer_size (integer, Minimum: 3, Maximum: 2048). Absolute value. Default is unbound. Doesn't work without indices.memory.index_buffer_size. Maximum amount of heap used for query cache, an absolute indices.memory.index_buffer_size maximum hard limit.indices_memory_min_index_buffer_size (integer, Minimum: 3, Maximum: 2048). Absolute value. Default is 48mb. Doesn't work without indices.memory.index_buffer_size. Minimum amount of heap used for query cache, an absolute indices.memory.index_buffer_size minimal hard limit.indices_queries_cache_size (integer, Minimum: 3, Maximum: 40). Percentage value. Default is 10%. Maximum amount of heap used for query cache. This is an expert setting. Too low value will decrease query performance and increase performance for other operations; too high value will cause issues with other OpenSearch functionality.indices_query_bool_max_clause_count (integer, Minimum: 64, Maximum: 4096). Maximum number of clauses Lucene BooleanQuery can have. The default value (1024) is relatively high, and increasing it may cause performance issues. Investigate other approaches first before increasing this value.indices_recovery_max_bytes_per_sec (integer, Minimum: 40, Maximum: 400). Limits total inbound and outbound recovery traffic for each node. Applies to both peer recoveries as well as snapshot recoveries (i.e., restores from a snapshot). Defaults to 40mb.indices_recovery_max_concurrent_file_chunks (integer, Minimum: 2, Maximum: 5). Number of file chunks sent in parallel for each recovery. Defaults to 2.ism_enabled (boolean). Specifies whether ISM is enabled or not.ism_history_enabled (boolean). Specifies whether audit history is enabled or not. The logs from ISM are automatically indexed to a logs document.ism_history_max_age (integer, Minimum: 1, Maximum: 2147483647). The maximum age before rolling over the audit history index in hours.ism_history_max_docs (integer, Minimum: 1). The maximum number of documents before rolling over the audit history index.ism_history_rollover_check_period (integer, Minimum: 1, Maximum: 2147483647). The time between rollover checks for the audit history index in hours.ism_history_rollover_retention_period (integer, Minimum: 1, Maximum: 2147483647). How long audit history indices are kept in days.override_main_response_version (boolean). Compatibility mode sets OpenSearch to report its version as 7.10 so clients continue to work. Default is false.plugins_alerting_filter_by_backend_roles (boolean). Enable or disable filtering of alerting by backend roles. Requires Security plugin. Defaults to false.reindex_remote_whitelist (array of strings, MaxItems: 32). Whitelisted addresses for reindexing. Changing this value will cause all OpenSearch instances to restart.script_max_compilations_rate (string, MaxLength: 1024). Script compilation circuit breaker limits the number of inline script compilations within a period of time. Default is use-context.search_max_buckets (integer, Minimum: 1, Maximum: 1000000). Maximum number of aggregation buckets allowed in a single response. OpenSearch default value is used when this is not defined.thread_pool_analyze_queue_size (integer, Minimum: 10, Maximum: 2000). Size for the thread pool queue. See documentation for exact details.thread_pool_analyze_size (integer, Minimum: 1, Maximum: 128). Size for the thread pool. See documentation for exact details. Do note this may have maximum value depending on CPU count - value is automatically lowered if set to higher than maximum value.thread_pool_force_merge_size (integer, Minimum: 1, Maximum: 128). Size for the thread pool. See documentation for exact details. Do note this may have maximum value depending on CPU count - value is automatically lowered if set to higher than maximum value.thread_pool_get_queue_size (integer, Minimum: 10, Maximum: 2000). Size for the thread pool queue. See documentation for exact details.thread_pool_get_size (integer, Minimum: 1, Maximum: 128). Size for the thread pool. See documentation for exact details. Do note this may have maximum value depending on CPU count - value is automatically lowered if set to higher than maximum value.thread_pool_search_queue_size (integer, Minimum: 10, Maximum: 2000). Size for the thread pool queue. See documentation for exact details.thread_pool_search_size (integer, Minimum: 1, Maximum: 128). Size for the thread pool. See documentation for exact details. Do note this may have maximum value depending on CPU count - value is automatically lowered if set to higher than maximum value.thread_pool_search_throttled_queue_size (integer, Minimum: 10, Maximum: 2000). Size for the thread pool queue. See documentation for exact details.thread_pool_search_throttled_size (integer, Minimum: 1, Maximum: 128). Size for the thread pool. See documentation for exact details. Do note this may have maximum value depending on CPU count - value is automatically lowered if set to higher than maximum value.thread_pool_write_queue_size (integer, Minimum: 10, Maximum: 2000). Size for the thread pool queue. See documentation for exact details.thread_pool_write_size (integer, Minimum: 1, Maximum: 128). Size for the thread pool. See documentation for exact details. Do note this may have maximum value depending on CPU count - value is automatically lowered if set to higher than maximum value.Appears on spec.userConfig.opensearch.
Opensearch Security Plugin Settings.
Optional
internal_authentication_backend_limiting (object). See below for nested schema.ip_rate_limiting (object). IP address rate limiting settings. See below for nested schema.Appears on spec.userConfig.opensearch.auth_failure_listeners.
Optional
allowed_tries (integer, Minimum: 0, Maximum: 2147483647). The number of login attempts allowed before login is blocked.authentication_backend (string, Enum: internal, MaxLength: 1024). internal_authentication_backend_limiting.authentication_backend.block_expiry_seconds (integer, Minimum: 0, Maximum: 2147483647). The duration of time that login remains blocked after a failed login.max_blocked_clients (integer, Minimum: 0, Maximum: 2147483647). internal_authentication_backend_limiting.max_blocked_clients.max_tracked_clients (integer, Minimum: 0, Maximum: 2147483647). The maximum number of tracked IP addresses that have failed login.time_window_seconds (integer, Minimum: 0, Maximum: 2147483647). The window of time in which the value for allowed_tries is enforced.type (string, Enum: username, MaxLength: 1024). internal_authentication_backend_limiting.type.Appears on spec.userConfig.opensearch.auth_failure_listeners.
IP address rate limiting settings.
Optional
allowed_tries (integer, Minimum: 1, Maximum: 2147483647). The number of login attempts allowed before login is blocked.block_expiry_seconds (integer, Minimum: 1, Maximum: 36000). The duration of time that login remains blocked after a failed login.max_blocked_clients (integer, Minimum: 0, Maximum: 2147483647). The maximum number of blocked IP addresses.max_tracked_clients (integer, Minimum: 0, Maximum: 2147483647). The maximum number of tracked IP addresses that have failed login.time_window_seconds (integer, Minimum: 1, Maximum: 36000). The window of time in which the value for allowed_tries is enforced.type (string, Enum: ip, MaxLength: 1024). The type of rate limiting.Appears on spec.userConfig.
OpenSearch Dashboards settings.
Optional
enabled (boolean). Enable or disable OpenSearch Dashboards.max_old_space_size (integer, Minimum: 64, Maximum: 2048). Limits the maximum amount of memory (in MiB) the OpenSearch Dashboards process can use. This sets the max_old_space_size option of the nodejs running the OpenSearch Dashboards. Note: the memory reserved by OpenSearch Dashboards is not available for OpenSearch.opensearch_request_timeout (integer, Minimum: 5000, Maximum: 120000). Timeout in milliseconds for requests made by OpenSearch Dashboards towards OpenSearch.Appears on spec.userConfig.
Allow access to selected service ports from private networks.
Optional
opensearch (boolean). Allow clients to connect to opensearch with a DNS name that always resolves to the service's private IP addresses. Only available in certain network locations.opensearch_dashboards (boolean). Allow clients to connect to opensearch_dashboards with a DNS name that always resolves to the service's private IP addresses. Only available in certain network locations.prometheus (boolean). Allow clients to connect to prometheus with a DNS name that always resolves to the service's private IP addresses. Only available in certain network locations.Appears on spec.userConfig.
Allow access to selected service components through Privatelink.
Optional
opensearch (boolean). Enable opensearch.opensearch_dashboards (boolean). Enable opensearch_dashboards.prometheus (boolean). Enable prometheus.Appears on spec.userConfig.
Allow access to selected service ports from the public Internet.
Optional
opensearch (boolean). Allow clients to connect to opensearch from the public internet for service nodes that are in a project VPC or another type of private network.opensearch_dashboards (boolean). Allow clients to connect to opensearch_dashboards from the public internet for service nodes that are in a project VPC or another type of private network.prometheus (boolean). Allow clients to connect to prometheus from the public internet for service nodes that are in a project VPC or another type of private network.Appears on spec.userConfig.
OpenSearch SAML configuration.
Required
enabled (boolean). Enables or disables SAML-based authentication for OpenSearch. When enabled, users can authenticate using SAML with an Identity Provider.idp_entity_id (string, Pattern: ^[^\\r\\n]*$, MinLength: 1, MaxLength: 1024). The unique identifier for the Identity Provider (IdP) entity that is used for SAML authentication. This value is typically provided by the IdP.idp_metadata_url (string, MinLength: 1, MaxLength: 2048). The URL of the SAML metadata for the Identity Provider (IdP). This is used to configure SAML-based authentication with the IdP.sp_entity_id (string, Pattern: ^[^\\r\\n]*$, MinLength: 1, MaxLength: 1024). The unique identifier for the Service Provider (SP) entity that is used for SAML authentication. This value is typically provided by the SP.Optional
idp_pemtrustedcas_content (string, MaxLength: 16384). This parameter specifies the PEM-encoded root certificate authority (CA) content for the SAML identity provider (IdP) server verification. The root CA content is used to verify the SSL/TLS certificate presented by the server.roles_key (string, Pattern: ^[^\\r\\n]*$, MinLength: 1, MaxLength: 256). Optional. Specifies the attribute in the SAML response where role information is stored, if available. Role attributes are not required for SAML authentication, but can be included in SAML assertions by most Identity Providers (IdPs) to determine user access levels or permissions.subject_key (string, Pattern: ^[^\\r\\n]*$, MinLength: 1, MaxLength: 256). Optional. Specifies the attribute in the SAML response where the subject identifier is stored. If not configured, the NameID attribute is used by default.apiVersion: aiven.io/v1alpha1\nkind: PostgreSQL\nmetadata:\n name: my-postgresql\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n connInfoSecretTarget:\n name: postgresql-secret\n prefix: MY_SECRET_PREFIX_\n annotations:\n foo: bar\n labels:\n baz: egg\n\n project: aiven-project-name\n cloudName: google-europe-west1\n plan: startup-4\n\n maintenanceWindowDow: sunday\n maintenanceWindowTime: 11:00:00\n\n userConfig:\n pg_version: \"15\"\nInfo
To create this resource, a Secret containing Aiven token must be created first.
Apply the resource with:
kubectl apply -f example.yaml\nVerify the newly created PostgreSQL:
kubectl get postgresqls my-postgresql\nThe output is similar to the following:
Name Project Region Plan State \nmy-postgresql aiven-project-name google-europe-west1 startup-4 RUNNING \nTo view the details of the Secret, use the following command:
kubectl describe secret postgresql-secret\nYou can use the jq to quickly decode the Secret:
kubectl get secret postgresql-secret -o json | jq '.data | map_values(@base64d)'\nThe output is similar to the following:
{\n \"POSTGRESQL_HOST\": \"<secret>\",\n \"POSTGRESQL_PORT\": \"<secret>\",\n \"POSTGRESQL_DATABASE\": \"<secret>\",\n \"POSTGRESQL_USER\": \"<secret>\",\n \"POSTGRESQL_PASSWORD\": \"<secret>\",\n \"POSTGRESQL_SSLMODE\": \"<secret>\",\n \"POSTGRESQL_DATABASE_URI\": \"<secret>\",\n \"POSTGRESQL_CA_CERT\": \"<secret>\",\n}\nPostgreSQL is the Schema for the postgresql API.
Exposes secret keys
POSTGRESQL_HOST, POSTGRESQL_PORT, POSTGRESQL_DATABASE, POSTGRESQL_USER, POSTGRESQL_PASSWORD, POSTGRESQL_SSLMODE, POSTGRESQL_DATABASE_URI, POSTGRESQL_CA_CERT.
Required
apiVersion (string). Value aiven.io/v1alpha1.kind (string). Value PostgreSQL.metadata (object). Data that identifies the object, including a name string and optional namespace.spec (object). PostgreSQLSpec defines the desired state of postgres instance. See below for nested schema.Appears on PostgreSQL.
PostgreSQLSpec defines the desired state of postgres instance.
Required
plan (string, MaxLength: 128). Subscription plan.project (string, Immutable, Pattern: ^[a-zA-Z0-9_-]+$, MaxLength: 63). Identifies the project this resource belongs to.Optional
authSecretRef (object). Authentication reference to Aiven token in a secret. See below for nested schema.cloudName (string, MaxLength: 256). Cloud the service runs in.connInfoSecretTarget (object). Secret configuration. See below for nested schema.connInfoSecretTargetDisabled (boolean, Immutable). When true, the secret containing connection information will not be created, defaults to false. This field cannot be changed after resource creation.disk_space (string, Pattern: (?i)^[1-9][0-9]*(GiB|G)?$). The disk space of the service, possible values depend on the service type, the cloud provider and the project. Reducing will result in the service re-balancing. The removal of this field does not change the value.maintenanceWindowDow (string, Enum: monday, tuesday, wednesday, thursday, friday, saturday, sunday). Day of week when maintenance operations should be performed. One monday, tuesday, wednesday, etc.maintenanceWindowTime (string, MaxLength: 8). Time of day when maintenance operations should be performed. UTC time in HH:mm:ss format.projectVPCRef (object). ProjectVPCRef reference to ProjectVPC resource to use its ID as ProjectVPCID automatically. See below for nested schema.projectVpcId (string, MaxLength: 36). Identifier of the VPC the service should be in, if any.serviceIntegrations (array of objects, Immutable, MaxItems: 1). Service integrations to specify when creating a service. Not applied after initial service creation. See below for nested schema.tags (object, AdditionalProperties: string). Tags are key-value pairs that allow you to categorize services.technicalEmails (array of objects, MaxItems: 10). Defines the email addresses that will receive alerts about upcoming maintenance updates or warnings about service instability. See below for nested schema.terminationProtection (boolean). Prevent service from being deleted. It is recommended to have this enabled for all services.userConfig (object). PostgreSQL specific user configuration options. See below for nested schema.Appears on spec.
Authentication reference to Aiven token in a secret.
Required
key (string, MinLength: 1).name (string, MinLength: 1).Appears on spec.
Secret configuration.
Required
name (string, Immutable). Name of the secret resource to be created. By default, it is equal to the resource name.Optional
annotations (object, AdditionalProperties: string). Annotations added to the secret.labels (object, AdditionalProperties: string). Labels added to the secret.prefix (string). Prefix for the secret's keys. Added \"as is\" without any transformations. By default, is equal to the kind name in uppercase + underscore, e.g. KAFKA_, REDIS_, etc.Appears on spec.
ProjectVPCRef reference to ProjectVPC resource to use its ID as ProjectVPCID automatically.
Required
name (string, MinLength: 1).Optional
namespace (string, MinLength: 1).Appears on spec.
Service integrations to specify when creating a service. Not applied after initial service creation.
Required
integrationType (string, Enum: read_replica).sourceServiceName (string, MinLength: 1, MaxLength: 64).Appears on spec.
Defines the email addresses that will receive alerts about upcoming maintenance updates or warnings about service instability.
Required
email (string). Email address.Appears on spec.
PostgreSQL specific user configuration options.
Optional
additional_backup_regions (array of strings, MaxItems: 1). Additional Cloud Regions for Backup Replication.admin_password (string, Immutable, Pattern: ^[a-zA-Z0-9-_]+$, MinLength: 8, MaxLength: 256). Custom password for admin user. Defaults to random string. This must be set only when a new service is being created.admin_username (string, Immutable, Pattern: ^[_A-Za-z0-9][-._A-Za-z0-9]{0,63}$, MaxLength: 64). Custom username for admin user. This must be set only when a new service is being created.backup_hour (integer, Minimum: 0, Maximum: 23). The hour of day (in UTC) when backup for the service is started. New backup is only started if previous backup has already completed.backup_minute (integer, Minimum: 0, Maximum: 59). The minute of an hour when backup for the service is started. New backup is only started if previous backup has already completed.enable_ipv6 (boolean). Register AAAA DNS records for the service, and allow IPv6 packets to service ports.ip_filter (array of objects, MaxItems: 1024). Allow incoming connections from CIDR address block, e.g. 10.20.0.0/16. See below for nested schema.migration (object). Migrate data from existing server. See below for nested schema.pg (object). postgresql.conf configuration values. See below for nested schema.pg_qualstats (object). Deprecated. System-wide settings for the pg_qualstats extension. See below for nested schema.pg_read_replica (boolean). Should the service which is being forked be a read replica (deprecated, use read_replica service integration instead).pg_service_to_fork_from (string, Immutable, Pattern: ^[a-z][-a-z0-9]{0,63}$|^$, MaxLength: 64). Name of the PG Service from which to fork (deprecated, use service_to_fork_from). This has effect only when a new service is being created.pg_stat_monitor_enable (boolean). Enable the pg_stat_monitor extension. Enabling this extension will cause the cluster to be restarted.When this extension is enabled, pg_stat_statements results for utility commands are unreliable.pg_version (string, Enum: 12, 13, 14, 15, 16). PostgreSQL major version.pgaudit (object). Deprecated. System-wide settings for the pgaudit extension. See below for nested schema.pgbouncer (object). PGBouncer connection pooling settings. See below for nested schema.pglookout (object). System-wide settings for pglookout. See below for nested schema.private_access (object). Allow access to selected service ports from private networks. See below for nested schema.privatelink_access (object). Allow access to selected service components through Privatelink. See below for nested schema.project_to_fork_from (string, Immutable, Pattern: ^[a-z][-a-z0-9]{0,63}$|^$, MaxLength: 63). Name of another project to fork a service from. This has effect only when a new service is being created.public_access (object). Allow access to selected service ports from the public Internet. See below for nested schema.recovery_target_time (string, Immutable, MaxLength: 32). Recovery target time when forking a service. This has effect only when a new service is being created.service_log (boolean). Store logs for the service so that they are available in the HTTP API and console.service_to_fork_from (string, Immutable, Pattern: ^[a-z][-a-z0-9]{0,63}$|^$, MaxLength: 64). Name of another service to fork from. This has effect only when a new service is being created.shared_buffers_percentage (number, Minimum: 20, Maximum: 60). Percentage of total RAM that the database server uses for shared memory buffers. Valid range is 20-60 (float), which corresponds to 20% - 60%. This setting adjusts the shared_buffers configuration value.static_ips (boolean). Use static public IP addresses.synchronous_replication (string, Enum: quorum, off). Synchronous replication type. Note that the service plan also needs to support synchronous replication.timescaledb (object). System-wide settings for the timescaledb extension. See below for nested schema.variant (string, Enum: aiven, timescale). Variant of the PostgreSQL service, may affect the features that are exposed by default.work_mem (integer, Minimum: 1, Maximum: 1024). Sets the maximum amount of memory to be used by a query operation (such as a sort or hash table) before writing to temporary disk files, in MB. Default is 1MB + 0.075% of total RAM (up to 32MB).Appears on spec.userConfig.
Allow incoming connections from CIDR address block, e.g. 10.20.0.0/16.
Required
network (string, MaxLength: 43). CIDR address block.Optional
description (string, MaxLength: 1024). Description for IP filter list entry.Appears on spec.userConfig.
Migrate data from existing server.
Required
host (string, MaxLength: 255). Hostname or IP address of the server where to migrate data from.port (integer, Minimum: 1, Maximum: 65535). Port number of the server where to migrate data from.Optional
dbname (string, MaxLength: 63). Database name for bootstrapping the initial connection.ignore_dbs (string, MaxLength: 2048). Comma-separated list of databases, which should be ignored during migration (supported by MySQL and PostgreSQL only at the moment).method (string, Enum: dump, replication). The migration method to be used (currently supported only by Redis, Dragonfly, MySQL and PostgreSQL service types).password (string, MaxLength: 256). Password for authentication with the server where to migrate data from.ssl (boolean). The server where to migrate data from is secured with SSL.username (string, MaxLength: 256). User name for authentication with the server where to migrate data from.Appears on spec.userConfig.
postgresql.conf configuration values.
Optional
autovacuum_analyze_scale_factor (number, Minimum: 0, Maximum: 1). Specifies a fraction of the table size to add to autovacuum_analyze_threshold when deciding whether to trigger an ANALYZE. The default is 0.2 (20% of table size).autovacuum_analyze_threshold (integer, Minimum: 0, Maximum: 2147483647). Specifies the minimum number of inserted, updated or deleted tuples needed to trigger an ANALYZE in any one table. The default is 50 tuples.autovacuum_freeze_max_age (integer, Minimum: 200000000, Maximum: 1500000000). Specifies the maximum age (in transactions) that a table's pg_class.relfrozenxid field can attain before a VACUUM operation is forced to prevent transaction ID wraparound within the table. Note that the system will launch autovacuum processes to prevent wraparound even when autovacuum is otherwise disabled. This parameter will cause the server to be restarted.autovacuum_max_workers (integer, Minimum: 1, Maximum: 20). Specifies the maximum number of autovacuum processes (other than the autovacuum launcher) that may be running at any one time. The default is three. This parameter can only be set at server start.autovacuum_naptime (integer, Minimum: 1, Maximum: 86400). Specifies the minimum delay between autovacuum runs on any given database. The delay is measured in seconds, and the default is one minute.autovacuum_vacuum_cost_delay (integer, Minimum: -1, Maximum: 100). Specifies the cost delay value that will be used in automatic VACUUM operations. If -1 is specified, the regular vacuum_cost_delay value will be used. The default value is 20 milliseconds.autovacuum_vacuum_cost_limit (integer, Minimum: -1, Maximum: 10000). Specifies the cost limit value that will be used in automatic VACUUM operations. If -1 is specified (which is the default), the regular vacuum_cost_limit value will be used.autovacuum_vacuum_scale_factor (number, Minimum: 0, Maximum: 1). Specifies a fraction of the table size to add to autovacuum_vacuum_threshold when deciding whether to trigger a VACUUM. The default is 0.2 (20% of table size).autovacuum_vacuum_threshold (integer, Minimum: 0, Maximum: 2147483647). Specifies the minimum number of updated or deleted tuples needed to trigger a VACUUM in any one table. The default is 50 tuples.bgwriter_delay (integer, Minimum: 10, Maximum: 10000). Specifies the delay between activity rounds for the background writer in milliseconds. Default is 200.bgwriter_flush_after (integer, Minimum: 0, Maximum: 2048). Whenever more than bgwriter_flush_after bytes have been written by the background writer, attempt to force the OS to issue these writes to the underlying storage. Specified in kilobytes, default is 512. Setting of 0 disables forced writeback.bgwriter_lru_maxpages (integer, Minimum: 0, Maximum: 1073741823). In each round, no more than this many buffers will be written by the background writer. Setting this to zero disables background writing. Default is 100.bgwriter_lru_multiplier (number, Minimum: 0, Maximum: 10). The average recent need for new buffers is multiplied by bgwriter_lru_multiplier to arrive at an estimate of the number that will be needed during the next round, (up to bgwriter_lru_maxpages). 1.0 represents a \u201cjust in time\u201d policy of writing exactly the number of buffers predicted to be needed. Larger values provide some cushion against spikes in demand, while smaller values intentionally leave writes to be done by server processes. The default is 2.0.deadlock_timeout (integer, Minimum: 500, Maximum: 1800000). This is the amount of time, in milliseconds, to wait on a lock before checking to see if there is a deadlock condition.default_toast_compression (string, Enum: lz4, pglz). Specifies the default TOAST compression method for values of compressible columns (the default is lz4).idle_in_transaction_session_timeout (integer, Minimum: 0, Maximum: 604800000). Time out sessions with open transactions after this number of milliseconds.jit (boolean). Controls system-wide use of Just-in-Time Compilation (JIT).log_autovacuum_min_duration (integer, Minimum: -1, Maximum: 2147483647). Causes each action executed by autovacuum to be logged if it ran for at least the specified number of milliseconds. Setting this to zero logs all autovacuum actions. Minus-one (the default) disables logging autovacuum actions.log_error_verbosity (string, Enum: TERSE, DEFAULT, VERBOSE). Controls the amount of detail written in the server log for each message that is logged.log_line_prefix (string, Enum: 'pid=%p,user=%u,db=%d,app=%a,client=%h ', '%t [%p]: [%l-1] user=%u,db=%d,app=%a,client=%h ', '%m [%p] %q[user=%u,db=%d,app=%a] '). Choose from one of the available log formats.log_min_duration_statement (integer, Minimum: -1, Maximum: 86400000). Log statements that take more than this number of milliseconds to run, -1 disables.log_temp_files (integer, Minimum: -1, Maximum: 2147483647). Log statements for each temporary file created larger than this number of kilobytes, -1 disables.max_files_per_process (integer, Minimum: 1000, Maximum: 4096). PostgreSQL maximum number of files that can be open per process.max_locks_per_transaction (integer, Minimum: 64, Maximum: 6400). PostgreSQL maximum locks per transaction.max_logical_replication_workers (integer, Minimum: 4, Maximum: 64). PostgreSQL maximum logical replication workers (taken from the pool of max_parallel_workers).max_parallel_workers (integer, Minimum: 0, Maximum: 96). Sets the maximum number of workers that the system can support for parallel queries.max_parallel_workers_per_gather (integer, Minimum: 0, Maximum: 96). Sets the maximum number of workers that can be started by a single Gather or Gather Merge node.max_pred_locks_per_transaction (integer, Minimum: 64, Maximum: 5120). PostgreSQL maximum predicate locks per transaction.max_prepared_transactions (integer, Minimum: 0, Maximum: 10000). PostgreSQL maximum prepared transactions.max_replication_slots (integer, Minimum: 8, Maximum: 64). PostgreSQL maximum replication slots.max_slot_wal_keep_size (integer, Minimum: -1, Maximum: 2147483647). PostgreSQL maximum WAL size (MB) reserved for replication slots. Default is -1 (unlimited). wal_keep_size minimum WAL size setting takes precedence over this.max_stack_depth (integer, Minimum: 2097152, Maximum: 6291456). Maximum depth of the stack in bytes.max_standby_archive_delay (integer, Minimum: 1, Maximum: 43200000). Max standby archive delay in milliseconds.max_standby_streaming_delay (integer, Minimum: 1, Maximum: 43200000). Max standby streaming delay in milliseconds.max_wal_senders (integer, Minimum: 20, Maximum: 64). PostgreSQL maximum WAL senders.max_worker_processes (integer, Minimum: 8, Maximum: 96). Sets the maximum number of background processes that the system can support.pg_partman_bgw.interval (integer, Minimum: 3600, Maximum: 604800). Sets the time interval to run pg_partman's scheduled tasks.pg_partman_bgw.role (string, Pattern: ^[_A-Za-z0-9][-._A-Za-z0-9]{0,63}$, MaxLength: 64). Controls which role to use for pg_partman's scheduled background tasks.pg_stat_monitor.pgsm_enable_query_plan (boolean). Enables or disables query plan monitoring.pg_stat_monitor.pgsm_max_buckets (integer, Minimum: 1, Maximum: 10). Sets the maximum number of buckets.pg_stat_statements.track (string, Enum: all, top, none). Controls which statements are counted. Specify top to track top-level statements (those issued directly by clients), all to also track nested statements (such as statements invoked within functions), or none to disable statement statistics collection. The default value is top.temp_file_limit (integer, Minimum: -1, Maximum: 2147483647). PostgreSQL temporary file limit in KiB, -1 for unlimited.timezone (string, Pattern: ^[\\w/]*$, MaxLength: 64). PostgreSQL service timezone.track_activity_query_size (integer, Minimum: 1024, Maximum: 10240). Specifies the number of bytes reserved to track the currently executing command for each active session.track_commit_timestamp (string, Enum: off, on). Record commit time of transactions.track_functions (string, Enum: all, pl, none). Enables tracking of function call counts and time used.track_io_timing (string, Enum: off, on). Enables timing of database I/O calls. This parameter is off by default, because it will repeatedly query the operating system for the current time, which may cause significant overhead on some platforms.wal_sender_timeout (integer). Terminate replication connections that are inactive for longer than this amount of time, in milliseconds. Setting this value to zero disables the timeout.wal_writer_delay (integer, Minimum: 10, Maximum: 200). WAL flush interval in milliseconds. Note that setting this value to lower than the default 200ms may negatively impact performance.Appears on spec.userConfig.
Deprecated. System-wide settings for the pg_qualstats extension.
Optional
enabled (boolean). Deprecated. Enable / Disable pg_qualstats.min_err_estimate_num (integer, Minimum: 0). Deprecated. Error estimation num threshold to save quals.min_err_estimate_ratio (integer, Minimum: 0). Deprecated. Error estimation ratio threshold to save quals.track_constants (boolean). Deprecated. Enable / Disable pg_qualstats constants tracking.track_pg_catalog (boolean). Deprecated. Track quals on system catalogs too.Appears on spec.userConfig.
Deprecated. System-wide settings for the pgaudit extension.
Optional
feature_enabled (boolean). Deprecated. Enable pgaudit extension. When enabled, pgaudit extension will be automatically installed.Otherwise, extension will be uninstalled but auditing configurations will be preserved.log (array of strings). Deprecated. Specifies which classes of statements will be logged by session audit logging.log_catalog (boolean). Deprecated. Specifies that session logging should be enabled in the casewhere all relations in a statement are in pg_catalog.log_client (boolean). Deprecated. Specifies whether log messages will be visible to a client process such as psql.log_level (string). Deprecated. Specifies the log level that will be used for log entries.log_max_string_length (integer, Minimum: -1, Maximum: 102400). Deprecated. Crop parameters representation and whole statements if they exceed this threshold. A (default) value of -1 disable the truncation.log_nested_statements (boolean). Deprecated. This GUC allows to turn off logging nested statements, that is, statements that are executed as part of another ExecutorRun.log_parameter (boolean). Deprecated. Specifies that audit logging should include the parameters that were passed with the statement.log_parameter_max_size (integer). Deprecated. Specifies that parameter values longer than this setting (in bytes) should not be logged, but replaced with . log_relation (boolean). Deprecated. Specifies whether session audit logging should create a separate log entry for each relation (TABLE, VIEW, etc.) referenced in a SELECT or DML statement.log_rows (boolean). Deprecated. Specifies that audit logging should include the rows retrieved or affected by a statement. When enabled the rows field will be included after the parameter field.log_statement (boolean). Deprecated. Specifies whether logging will include the statement text and parameters (if enabled).log_statement_once (boolean). Deprecated. Specifies whether logging will include the statement text and parameters with the first log entry for a statement/substatement combination or with every entry.role (string, Pattern: ^[_A-Za-z0-9][-._A-Za-z0-9]{0,63}$, MaxLength: 64). Deprecated. Specifies the master role to use for object audit logging.Appears on spec.userConfig.
PGBouncer connection pooling settings.
Optional
autodb_idle_timeout (integer, Minimum: 0, Maximum: 86400). If the automatically created database pools have been unused this many seconds, they are freed. If 0 then timeout is disabled. [seconds].autodb_max_db_connections (integer, Minimum: 0, Maximum: 2147483647). Do not allow more than this many server connections per database (regardless of user). Setting it to 0 means unlimited.autodb_pool_mode (string, Enum: session, transaction, statement). PGBouncer pool mode.autodb_pool_size (integer, Minimum: 0, Maximum: 10000). If non-zero then create automatically a pool of that size per user when a pool doesn't exist.ignore_startup_parameters (array of strings, MaxItems: 32). List of parameters to ignore when given in startup packet.min_pool_size (integer, Minimum: 0, Maximum: 10000). Add more server connections to pool if below this number. Improves behavior when usual load comes suddenly back after period of total inactivity. The value is effectively capped at the pool size.server_idle_timeout (integer, Minimum: 0, Maximum: 86400). If a server connection has been idle more than this many seconds it will be dropped. If 0 then timeout is disabled. [seconds].server_lifetime (integer, Minimum: 60, Maximum: 86400). The pooler will close an unused server connection that has been connected longer than this. [seconds].server_reset_query_always (boolean). Run server_reset_query (DISCARD ALL) in all pooling modes.Appears on spec.userConfig.
System-wide settings for pglookout.
Required
max_failover_replication_time_lag (integer, Minimum: 10). Number of seconds of master unavailability before triggering database failover to standby.Appears on spec.userConfig.
Allow access to selected service ports from private networks.
Optional
pg (boolean). Allow clients to connect to pg with a DNS name that always resolves to the service's private IP addresses. Only available in certain network locations.pgbouncer (boolean). Allow clients to connect to pgbouncer with a DNS name that always resolves to the service's private IP addresses. Only available in certain network locations.prometheus (boolean). Allow clients to connect to prometheus with a DNS name that always resolves to the service's private IP addresses. Only available in certain network locations.Appears on spec.userConfig.
Allow access to selected service components through Privatelink.
Optional
pg (boolean). Enable pg.pgbouncer (boolean). Enable pgbouncer.prometheus (boolean). Enable prometheus.Appears on spec.userConfig.
Allow access to selected service ports from the public Internet.
Optional
pg (boolean). Allow clients to connect to pg from the public internet for service nodes that are in a project VPC or another type of private network.pgbouncer (boolean). Allow clients to connect to pgbouncer from the public internet for service nodes that are in a project VPC or another type of private network.prometheus (boolean). Allow clients to connect to prometheus from the public internet for service nodes that are in a project VPC or another type of private network.Appears on spec.userConfig.
System-wide settings for the timescaledb extension.
Required
max_background_workers (integer, Minimum: 1, Maximum: 4096). The number of background workers for timescaledb operations. You should configure this setting to the sum of your number of databases and the total number of concurrent background workers you want running at any given point in time.apiVersion: aiven.io/v1alpha1\nkind: Project\nmetadata:\n name: my-project\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n connInfoSecretTarget:\n name: project-secret\n prefix: MY_SECRET_PREFIX_\n annotations:\n foo: bar\n labels:\n baz: egg\n\n tags:\n env: prod\n\n billingAddress: NYC\n cloud: aws-eu-west-1\nInfo
To create this resource, a Secret containing Aiven token must be created first.
Apply the resource with:
kubectl apply -f example.yaml\nVerify the newly created Project:
kubectl get projects my-project\nThe output is similar to the following:
Name \nmy-project \nTo view the details of the Secret, use the following command:
kubectl describe secret project-secret\nYou can use the jq to quickly decode the Secret:
kubectl get secret project-secret -o json | jq '.data | map_values(@base64d)'\nThe output is similar to the following:
{\n \"PROJECT_CA_CERT\": \"<secret>\",\n}\nProject is the Schema for the projects API.
Exposes secret keys
PROJECT_CA_CERT.
Required
apiVersion (string). Value aiven.io/v1alpha1.kind (string). Value Project.metadata (object). Data that identifies the object, including a name string and optional namespace.spec (object). ProjectSpec defines the desired state of Project. See below for nested schema.Appears on Project.
ProjectSpec defines the desired state of Project.
Optional
accountId (string, MaxLength: 32). Account ID.authSecretRef (object). Authentication reference to Aiven token in a secret. See below for nested schema.billingAddress (string, MaxLength: 1000). Billing name and address of the project.billingCurrency (string, Enum: AUD, CAD, CHF, DKK, EUR, GBP, NOK, SEK, USD). Billing currency.billingEmails (array of strings, MaxItems: 10). Billing contact emails of the project.billingExtraText (string, MaxLength: 1000). Extra text to be included in all project invoices, e.g. purchase order or cost center number.billingGroupId (string, Immutable, MinLength: 36, MaxLength: 36). BillingGroup ID.cardId (string, MaxLength: 64). Credit card ID; The ID may be either last 4 digits of the card or the actual ID.cloud (string, MaxLength: 256). Target cloud, example: aws-eu-central-1.connInfoSecretTarget (object). Secret configuration. See below for nested schema.connInfoSecretTargetDisabled (boolean, Immutable). When true, the secret containing connection information will not be created, defaults to false. This field cannot be changed after resource creation.copyFromProject (string, Immutable, MaxLength: 63). Project name from which to copy settings to the new project.countryCode (string, MinLength: 2, MaxLength: 2). Billing country code of the project.tags (object, AdditionalProperties: string). Tags are key-value pairs that allow you to categorize projects.technicalEmails (array of strings, MaxItems: 10). Technical contact emails of the project.Appears on spec.
Authentication reference to Aiven token in a secret.
Required
key (string, MinLength: 1).name (string, MinLength: 1).Appears on spec.
Secret configuration.
Required
name (string, Immutable). Name of the secret resource to be created. By default, it is equal to the resource name.Optional
annotations (object, AdditionalProperties: string). Annotations added to the secret.labels (object, AdditionalProperties: string). Labels added to the secret.prefix (string). Prefix for the secret's keys. Added \"as is\" without any transformations. By default, is equal to the kind name in uppercase + underscore, e.g. KAFKA_, REDIS_, etc.apiVersion: aiven.io/v1alpha1\nkind: ProjectVPC\nmetadata:\n name: my-project-vpc\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n project: aiven-project-name\n cloudName: google-europe-west1\n networkCidr: 10.0.0.0/24\nInfo
To create this resource, a Secret containing Aiven token must be created first.
Apply the resource with:
kubectl apply -f example.yaml\nVerify the newly created ProjectVPC:
kubectl get projectvpcs my-project-vpc\nThe output is similar to the following:
Name Project Cloud Network CIDR State \nmy-project-vpc aiven-project-name google-europe-west1 10.0.0.0/24 RUNNING \nProjectVPC is the Schema for the projectvpcs API.
Required
apiVersion (string). Value aiven.io/v1alpha1.kind (string). Value ProjectVPC.metadata (object). Data that identifies the object, including a name string and optional namespace.spec (object). ProjectVPCSpec defines the desired state of ProjectVPC. See below for nested schema.Appears on ProjectVPC.
ProjectVPCSpec defines the desired state of ProjectVPC.
Required
cloudName (string, Immutable, MaxLength: 256). Cloud the VPC is in.networkCidr (string, Immutable, MaxLength: 36). Network address range used by the VPC like 192.168.0.0/24.project (string, Immutable, Pattern: ^[a-zA-Z0-9_-]+$, MaxLength: 63). Identifies the project this resource belongs to.Optional
authSecretRef (object). Authentication reference to Aiven token in a secret. See below for nested schema.Appears on spec.
Authentication reference to Aiven token in a secret.
Required
key (string, MinLength: 1).name (string, MinLength: 1).apiVersion: aiven.io/v1alpha1\nkind: Redis\nmetadata:\n name: k8s-redis\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n connInfoSecretTarget:\n name: redis-token\n prefix: MY_SECRET_PREFIX_\n annotations:\n foo: bar\n labels:\n baz: egg\n\n project: my-aiven-project\n cloudName: google-europe-west1\n plan: startup-4\n\n maintenanceWindowDow: friday\n maintenanceWindowTime: 23:00:00\n\n userConfig:\n redis_maxmemory_policy: allkeys-random\nInfo
To create this resource, a Secret containing Aiven token must be created first.
Apply the resource with:
kubectl apply -f example.yaml\nVerify the newly created Redis:
kubectl get redis k8s-redis\nThe output is similar to the following:
Name Project Region Plan State \nk8s-redis my-aiven-project google-europe-west1 startup-4 RUNNING \nTo view the details of the Secret, use the following command:
kubectl describe secret redis-token\nYou can use the jq to quickly decode the Secret:
kubectl get secret redis-token -o json | jq '.data | map_values(@base64d)'\nThe output is similar to the following:
{\n \"REDIS_HOST\": \"<secret>\",\n \"REDIS_PORT\": \"<secret>\",\n \"REDIS_USER\": \"<secret>\",\n \"REDIS_PASSWORD\": \"<secret>\",\n}\nRedis is the Schema for the redis API.
Exposes secret keys
REDIS_HOST, REDIS_PORT, REDIS_USER, REDIS_PASSWORD.
Required
apiVersion (string). Value aiven.io/v1alpha1.kind (string). Value Redis.metadata (object). Data that identifies the object, including a name string and optional namespace.spec (object). RedisSpec defines the desired state of Redis. See below for nested schema.Appears on Redis.
RedisSpec defines the desired state of Redis.
Required
plan (string, MaxLength: 128). Subscription plan.project (string, Immutable, Pattern: ^[a-zA-Z0-9_-]+$, MaxLength: 63). Identifies the project this resource belongs to.Optional
authSecretRef (object). Authentication reference to Aiven token in a secret. See below for nested schema.cloudName (string, MaxLength: 256). Cloud the service runs in.connInfoSecretTarget (object). Secret configuration. See below for nested schema.connInfoSecretTargetDisabled (boolean, Immutable). When true, the secret containing connection information will not be created, defaults to false. This field cannot be changed after resource creation.disk_space (string, Pattern: (?i)^[1-9][0-9]*(GiB|G)?$). The disk space of the service, possible values depend on the service type, the cloud provider and the project. Reducing will result in the service re-balancing. The removal of this field does not change the value.maintenanceWindowDow (string, Enum: monday, tuesday, wednesday, thursday, friday, saturday, sunday). Day of week when maintenance operations should be performed. One monday, tuesday, wednesday, etc.maintenanceWindowTime (string, MaxLength: 8). Time of day when maintenance operations should be performed. UTC time in HH:mm:ss format.projectVPCRef (object). ProjectVPCRef reference to ProjectVPC resource to use its ID as ProjectVPCID automatically. See below for nested schema.projectVpcId (string, MaxLength: 36). Identifier of the VPC the service should be in, if any.serviceIntegrations (array of objects, Immutable, MaxItems: 1). Service integrations to specify when creating a service. Not applied after initial service creation. See below for nested schema.tags (object, AdditionalProperties: string). Tags are key-value pairs that allow you to categorize services.technicalEmails (array of objects, MaxItems: 10). Defines the email addresses that will receive alerts about upcoming maintenance updates or warnings about service instability. See below for nested schema.terminationProtection (boolean). Prevent service from being deleted. It is recommended to have this enabled for all services.userConfig (object). Redis specific user configuration options. See below for nested schema.Appears on spec.
Authentication reference to Aiven token in a secret.
Required
key (string, MinLength: 1).name (string, MinLength: 1).Appears on spec.
Secret configuration.
Required
name (string, Immutable). Name of the secret resource to be created. By default, it is equal to the resource name.Optional
annotations (object, AdditionalProperties: string). Annotations added to the secret.labels (object, AdditionalProperties: string). Labels added to the secret.prefix (string). Prefix for the secret's keys. Added \"as is\" without any transformations. By default, is equal to the kind name in uppercase + underscore, e.g. KAFKA_, REDIS_, etc.Appears on spec.
ProjectVPCRef reference to ProjectVPC resource to use its ID as ProjectVPCID automatically.
Required
name (string, MinLength: 1).Optional
namespace (string, MinLength: 1).Appears on spec.
Service integrations to specify when creating a service. Not applied after initial service creation.
Required
integrationType (string, Enum: read_replica).sourceServiceName (string, MinLength: 1, MaxLength: 64).Appears on spec.
Defines the email addresses that will receive alerts about upcoming maintenance updates or warnings about service instability.
Required
email (string). Email address.Appears on spec.
Redis specific user configuration options.
Optional
additional_backup_regions (array of strings, MaxItems: 1). Additional Cloud Regions for Backup Replication.ip_filter (array of objects, MaxItems: 1024). Allow incoming connections from CIDR address block, e.g. 10.20.0.0/16. See below for nested schema.migration (object). Migrate data from existing server. See below for nested schema.private_access (object). Allow access to selected service ports from private networks. See below for nested schema.privatelink_access (object). Allow access to selected service components through Privatelink. See below for nested schema.project_to_fork_from (string, Immutable, Pattern: ^[a-z][-a-z0-9]{0,63}$|^$, MaxLength: 63). Name of another project to fork a service from. This has effect only when a new service is being created.public_access (object). Allow access to selected service ports from the public Internet. See below for nested schema.recovery_basebackup_name (string, Pattern: ^[a-zA-Z0-9-_:.]+$, MaxLength: 128). Name of the basebackup to restore in forked service.redis_acl_channels_default (string, Enum: allchannels, resetchannels). Determines default pub/sub channels' ACL for new users if ACL is not supplied. When this option is not defined, all_channels is assumed to keep backward compatibility. This option doesn't affect Redis configuration acl-pubsub-default.redis_io_threads (integer, Minimum: 1, Maximum: 32). Set Redis IO thread count. Changing this will cause a restart of the Redis service.redis_lfu_decay_time (integer, Minimum: 1, Maximum: 120). LFU maxmemory-policy counter decay time in minutes.redis_lfu_log_factor (integer, Minimum: 0, Maximum: 100). Counter logarithm factor for volatile-lfu and allkeys-lfu maxmemory-policies.redis_maxmemory_policy (string, Enum: noeviction, allkeys-lru, volatile-lru, allkeys-random, volatile-random, volatile-ttl, volatile-lfu, allkeys-lfu). Redis maxmemory-policy.redis_notify_keyspace_events (string, Pattern: ^[KEg\\$lshzxentdmA]*$, MaxLength: 32). Set notify-keyspace-events option.redis_number_of_databases (integer, Minimum: 1, Maximum: 128). Set number of Redis databases. Changing this will cause a restart of the Redis service.redis_persistence (string, Enum: off, rdb). When persistence is rdb, Redis does RDB dumps each 10 minutes if any key is changed. Also RDB dumps are done according to the backup schedule for backup purposes. When persistence is off, no RDB dumps or backups are done, so data can be lost at any moment if the service is restarted for any reason, or if the service is powered off. Also, the service can't be forked.redis_pubsub_client_output_buffer_limit (integer, Minimum: 32, Maximum: 512). Set output buffer limit for pub / sub clients in MB. The value is the hard limit, the soft limit is 1/4 of the hard limit. When setting the limit, be mindful of the available memory in the selected service plan.redis_ssl (boolean). Require SSL to access Redis.redis_timeout (integer, Minimum: 0, Maximum: 31536000). Redis idle connection timeout in seconds.redis_version (string, Enum: 7.0). Redis major version.service_log (boolean). Store logs for the service so that they are available in the HTTP API and console.service_to_fork_from (string, Immutable, Pattern: ^[a-z][-a-z0-9]{0,63}$|^$, MaxLength: 64). Name of another service to fork from. This has effect only when a new service is being created.static_ips (boolean). Use static public IP addresses.Appears on spec.userConfig.
Allow incoming connections from CIDR address block, e.g. 10.20.0.0/16.
Required
network (string, MaxLength: 43). CIDR address block.Optional
description (string, MaxLength: 1024). Description for IP filter list entry.Appears on spec.userConfig.
Migrate data from existing server.
Required
host (string, MaxLength: 255). Hostname or IP address of the server where to migrate data from.port (integer, Minimum: 1, Maximum: 65535). Port number of the server where to migrate data from.Optional
dbname (string, MaxLength: 63). Database name for bootstrapping the initial connection.ignore_dbs (string, MaxLength: 2048). Comma-separated list of databases, which should be ignored during migration (supported by MySQL and PostgreSQL only at the moment).method (string, Enum: dump, replication). The migration method to be used (currently supported only by Redis, Dragonfly, MySQL and PostgreSQL service types).password (string, MaxLength: 256). Password for authentication with the server where to migrate data from.ssl (boolean). The server where to migrate data from is secured with SSL.username (string, MaxLength: 256). User name for authentication with the server where to migrate data from.Appears on spec.userConfig.
Allow access to selected service ports from private networks.
Optional
prometheus (boolean). Allow clients to connect to prometheus with a DNS name that always resolves to the service's private IP addresses. Only available in certain network locations.redis (boolean). Allow clients to connect to redis with a DNS name that always resolves to the service's private IP addresses. Only available in certain network locations.Appears on spec.userConfig.
Allow access to selected service components through Privatelink.
Optional
prometheus (boolean). Enable prometheus.redis (boolean). Enable redis.Appears on spec.userConfig.
Allow access to selected service ports from the public Internet.
Optional
prometheus (boolean). Allow clients to connect to prometheus from the public internet for service nodes that are in a project VPC or another type of private network.redis (boolean). Allow clients to connect to redis from the public internet for service nodes that are in a project VPC or another type of private network.apiVersion: aiven.io/v1alpha1\nkind: ServiceIntegration\nmetadata:\n name: my-service-integration\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n project: aiven-project-name\n integrationType: clickhouse_postgresql\n sourceServiceName: my-pg\n destinationServiceName: my-clickhouse\n\n clickhousePostgresql:\n databases:\n - database: defaultdb\n schema: public\n\n---\n\napiVersion: aiven.io/v1alpha1\nkind: Clickhouse\nmetadata:\n name: my-clickhouse\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n project: aiven-project-name\n cloudName: google-europe-west1\n plan: startup-16\n maintenanceWindowDow: friday\n maintenanceWindowTime: 23:00:00\n\n---\n\napiVersion: aiven.io/v1alpha1\nkind: PostgreSQL\nmetadata:\n name: my-pg\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n project: aiven-project-name\n cloudName: google-europe-west1\n plan: startup-4\n maintenanceWindowDow: friday\n maintenanceWindowTime: 23:00:00\napiVersion: aiven.io/v1alpha1\nkind: ServiceIntegration\nmetadata:\n name: my-service-integration\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n project: aiven-project-name\n integrationType: datadog\n sourceServiceName: my-pg\n destinationEndpointId: destination-endpoint-id\n\n datadog:\n datadog_dbm_enabled: True\n datadog_tags:\n - tag: env\n comment: test\n\n---\n\napiVersion: aiven.io/v1alpha1\nkind: PostgreSQL\nmetadata:\n name: my-pg\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n project: aiven-project-name\n cloudName: google-europe-west1\n plan: startup-4\napiVersion: aiven.io/v1alpha1\nkind: ServiceIntegration\nmetadata:\n name: my-service-integration\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n project: aiven-project-name\n integrationType: kafka_connect\n sourceServiceName: my-kafka\n destinationServiceName: my-kafka-connect\n\n kafkaConnect:\n kafka_connect:\n group_id: connect\n status_storage_topic: __connect_status\n offset_storage_topic: __connect_offsets\n\n---\n\napiVersion: aiven.io/v1alpha1\nkind: Kafka\nmetadata:\n name: my-kafka\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n project: aiven-project-name\n cloudName: google-europe-west1\n plan: business-4\n\n---\n\napiVersion: aiven.io/v1alpha1\nkind: KafkaConnect\nmetadata:\n name: my-kafka-connect\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n project: aiven-project-name\n cloudName: google-europe-west1\n plan: business-4\n\n userConfig:\n kafka_connect:\n consumer_isolation_level: read_committed\n public_access:\n kafka_connect: true\napiVersion: aiven.io/v1alpha1\nkind: ServiceIntegration\nmetadata:\n name: my-service-integration\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n project: aiven-project-name\n integrationType: kafka_logs\n sourceServiceName: my-kafka\n destinationServiceName: my-kafka\n\n kafkaLogs:\n kafka_topic: my-kafka-topic\n\n---\n\napiVersion: aiven.io/v1alpha1\nkind: Kafka\nmetadata:\n name: my-kafka\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n project: aiven-project-name\n cloudName: google-europe-west1\n plan: business-4\n\n---\n\napiVersion: aiven.io/v1alpha1\nkind: KafkaTopic\nmetadata:\n name: my-kafka-topic\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n project: aiven-project-name\n serviceName: my-kafka\n replication: 2\n partitions: 1\nInfo
To create this resource, a Secret containing Aiven token must be created first.
Apply the resource with:
kubectl apply -f example.yaml\nVerify the newly created ServiceIntegration:
kubectl get serviceintegrations my-service-integration\nThe output is similar to the following:
Name Project Type Source Service Name Destination Service Name \nmy-service-integration aiven-project-name clickhouse_postgresql my-pg my-clickhouse \nServiceIntegration is the Schema for the serviceintegrations API.
Required
apiVersion (string). Value aiven.io/v1alpha1.kind (string). Value ServiceIntegration.metadata (object). Data that identifies the object, including a name string and optional namespace.spec (object). ServiceIntegrationSpec defines the desired state of ServiceIntegration. See below for nested schema.Appears on ServiceIntegration.
ServiceIntegrationSpec defines the desired state of ServiceIntegration.
Required
integrationType (string, Enum: alertmanager, autoscaler, caching, cassandra_cross_service_cluster, clickhouse_kafka, clickhouse_postgresql, dashboard, datadog, datasource, external_aws_cloudwatch_logs, external_aws_cloudwatch_metrics, external_elasticsearch_logs, external_google_cloud_logging, external_opensearch_logs, flink, flink_external_kafka, flink_external_postgresql, internal_connectivity, jolokia, kafka_connect, kafka_logs, kafka_mirrormaker, logs, m3aggregator, m3coordinator, metrics, opensearch_cross_cluster_replication, opensearch_cross_cluster_search, prometheus, read_replica, rsyslog, schema_registry_proxy, stresstester, thanosquery, thanosstore, vmalert, Immutable). Type of the service integration accepted by Aiven API. Some values may not be supported by the operator.project (string, Immutable, Pattern: ^[a-zA-Z0-9_-]+$, MaxLength: 63). Identifies the project this resource belongs to.Optional
authSecretRef (object). Authentication reference to Aiven token in a secret. See below for nested schema.clickhouseKafka (object). Clickhouse Kafka configuration values. See below for nested schema.clickhousePostgresql (object). Clickhouse PostgreSQL configuration values. See below for nested schema.datadog (object). Datadog specific user configuration options. See below for nested schema.destinationEndpointId (string, Immutable, MaxLength: 36). Destination endpoint for the integration (if any).destinationProjectName (string, Immutable, MaxLength: 63). Destination project for the integration (if any).destinationServiceName (string, Immutable, MaxLength: 64). Destination service for the integration (if any).externalAWSCloudwatchMetrics (object). External AWS CloudWatch Metrics integration Logs configuration values. See below for nested schema.kafkaConnect (object). Kafka Connect service configuration values. See below for nested schema.kafkaLogs (object). Kafka logs configuration values. See below for nested schema.kafkaMirrormaker (object). Kafka MirrorMaker configuration values. See below for nested schema.logs (object). Logs configuration values. See below for nested schema.metrics (object). Metrics configuration values. See below for nested schema.sourceEndpointID (string, Immutable, MaxLength: 36). Source endpoint for the integration (if any).sourceProjectName (string, Immutable, MaxLength: 63). Source project for the integration (if any).sourceServiceName (string, Immutable, MaxLength: 64). Source service for the integration (if any).Appears on spec.
Authentication reference to Aiven token in a secret.
Required
key (string, MinLength: 1).name (string, MinLength: 1).Appears on spec.
Clickhouse Kafka configuration values.
Required
tables (array of objects, MaxItems: 100). Tables to create. See below for nested schema.Appears on spec.clickhouseKafka.
Tables to create.
Required
columns (array of objects, MaxItems: 100). Table columns. See below for nested schema.data_format (string, Enum: Avro, CSV, JSONAsString, JSONCompactEachRow, JSONCompactStringsEachRow, JSONEachRow, JSONStringsEachRow, MsgPack, TSKV, TSV, TabSeparated, RawBLOB, AvroConfluent, Parquet). Message data format.group_name (string, MinLength: 1, MaxLength: 249). Kafka consumers group.name (string, MinLength: 1, MaxLength: 40). Name of the table.topics (array of objects, MaxItems: 100). Kafka topics. See below for nested schema.Optional
auto_offset_reset (string, Enum: smallest, earliest, beginning, largest, latest, end). Action to take when there is no initial offset in offset store or the desired offset is out of range.date_time_input_format (string, Enum: basic, best_effort, best_effort_us). Method to read DateTime from text input formats.handle_error_mode (string, Enum: default, stream). How to handle errors for Kafka engine.max_block_size (integer, Minimum: 0, Maximum: 1000000000). Number of row collected by poll(s) for flushing data from Kafka.max_rows_per_message (integer, Minimum: 1, Maximum: 1000000000). The maximum number of rows produced in one kafka message for row-based formats.num_consumers (integer, Minimum: 1, Maximum: 10). The number of consumers per table per replica.poll_max_batch_size (integer, Minimum: 0, Maximum: 1000000000). Maximum amount of messages to be polled in a single Kafka poll.skip_broken_messages (integer, Minimum: 0, Maximum: 1000000000). Skip at least this number of broken messages from Kafka topic per block.Appears on spec.clickhouseKafka.tables.
Table columns.
Required
name (string, MinLength: 1, MaxLength: 40). Column name.type (string, MinLength: 1, MaxLength: 1000). Column type.Appears on spec.clickhouseKafka.tables.
Kafka topics.
Required
name (string, MinLength: 1, MaxLength: 249). Name of the topic.Appears on spec.
Clickhouse PostgreSQL configuration values.
Required
databases (array of objects, MaxItems: 10). Databases to expose. See below for nested schema.Appears on spec.clickhousePostgresql.
Databases to expose.
Optional
database (string, MinLength: 1, MaxLength: 63). PostgreSQL database to expose.schema (string, MinLength: 1, MaxLength: 63). PostgreSQL schema to expose.Appears on spec.
Datadog specific user configuration options.
Optional
datadog_dbm_enabled (boolean). Enable Datadog Database Monitoring.datadog_pgbouncer_enabled (boolean). Enable Datadog PgBouncer Metric Tracking.datadog_tags (array of objects, MaxItems: 32). Custom tags provided by user. See below for nested schema.exclude_consumer_groups (array of strings, MaxItems: 1024). List of custom metrics.exclude_topics (array of strings, MaxItems: 1024). List of topics to exclude.include_consumer_groups (array of strings, MaxItems: 1024). List of custom metrics.include_topics (array of strings, MaxItems: 1024). List of topics to include.kafka_custom_metrics (array of strings, MaxItems: 1024). List of custom metrics.max_jmx_metrics (integer, Minimum: 10, Maximum: 100000). Maximum number of JMX metrics to send.mirrormaker_custom_metrics (array of strings, MaxItems: 1024). List of custom metrics.opensearch (object). Datadog Opensearch Options. See below for nested schema.redis (object). Datadog Redis Options. See below for nested schema.Appears on spec.datadog.
Custom tags provided by user.
Required
tag (string, MinLength: 1, MaxLength: 200). Tag format and usage are described here: https://docs.datadoghq.com/getting_started/tagging. Tags with prefix aiven- are reserved for Aiven.Optional
comment (string, MaxLength: 1024). Optional tag explanation.Appears on spec.datadog.
Datadog Opensearch Options.
Optional
cluster_stats_enabled (boolean). Enable Datadog Opensearch Cluster Monitoring.index_stats_enabled (boolean). Enable Datadog Opensearch Index Monitoring.pending_task_stats_enabled (boolean). Enable Datadog Opensearch Pending Task Monitoring.pshard_stats_enabled (boolean). Enable Datadog Opensearch Primary Shard Monitoring.Appears on spec.datadog.
Datadog Redis Options.
Required
command_stats_enabled (boolean). Enable command_stats option in the agent's configuration.Appears on spec.
External AWS CloudWatch Metrics integration Logs configuration values.
Optional
dropped_metrics (array of objects, MaxItems: 1024). Metrics to not send to AWS CloudWatch (takes precedence over extra_metrics). See below for nested schema.extra_metrics (array of objects, MaxItems: 1024). Metrics to allow through to AWS CloudWatch (in addition to default metrics). See below for nested schema.Appears on spec.externalAWSCloudwatchMetrics.
Metrics to not send to AWS CloudWatch (takes precedence over extra_metrics).
Required
field (string, MaxLength: 1000). Identifier of a value in the metric.metric (string, MaxLength: 1000). Identifier of the metric.Appears on spec.externalAWSCloudwatchMetrics.
Metrics to allow through to AWS CloudWatch (in addition to default metrics).
Required
field (string, MaxLength: 1000). Identifier of a value in the metric.metric (string, MaxLength: 1000). Identifier of the metric.Appears on spec.
Kafka Connect service configuration values.
Required
kafka_connect (object). Kafka Connect service configuration values. See below for nested schema.Appears on spec.kafkaConnect.
Kafka Connect service configuration values.
Optional
config_storage_topic (string, MaxLength: 249). The name of the topic where connector and task configuration data are stored.This must be the same for all workers with the same group_id.group_id (string, MaxLength: 249). A unique string that identifies the Connect cluster group this worker belongs to.offset_storage_topic (string, MaxLength: 249). The name of the topic where connector and task configuration offsets are stored.This must be the same for all workers with the same group_id.status_storage_topic (string, MaxLength: 249). The name of the topic where connector and task configuration status updates are stored.This must be the same for all workers with the same group_id.Appears on spec.
Kafka logs configuration values.
Required
kafka_topic (string, MinLength: 1, MaxLength: 249). Topic name.Optional
selected_log_fields (array of strings, MaxItems: 5). The list of logging fields that will be sent to the integration logging service. The MESSAGE and timestamp fields are always sent.Appears on spec.
Kafka MirrorMaker configuration values.
Optional
cluster_alias (string, Pattern: ^[a-zA-Z0-9_.-]+$, MaxLength: 128). The alias under which the Kafka cluster is known to MirrorMaker. Can contain the following symbols: ASCII alphanumerics, ., _, and -.kafka_mirrormaker (object). Kafka MirrorMaker configuration values. See below for nested schema.Appears on spec.kafkaMirrormaker.
Kafka MirrorMaker configuration values.
Optional
consumer_auto_offset_reset (string, Enum: earliest, latest). Set where consumer starts to consume data. Value earliest: Start replication from the earliest offset. Value latest: Start replication from the latest offset. Default is earliest.consumer_fetch_min_bytes (integer, Minimum: 1, Maximum: 5242880). The minimum amount of data the server should return for a fetch request.consumer_max_poll_records (integer, Minimum: 100, Maximum: 20000). Set consumer max.poll.records. The default is 500.producer_batch_size (integer, Minimum: 0, Maximum: 5242880). The batch size in bytes producer will attempt to collect before publishing to broker.producer_buffer_memory (integer, Minimum: 5242880, Maximum: 134217728). The amount of bytes producer can use for buffering data before publishing to broker.producer_compression_type (string, Enum: gzip, snappy, lz4, zstd, none). Specify the default compression type for producers. This configuration accepts the standard compression codecs (gzip, snappy, lz4, zstd). It additionally accepts none which is the default and equivalent to no compression.producer_linger_ms (integer, Minimum: 0, Maximum: 5000). The linger time (ms) for waiting new data to arrive for publishing.producer_max_request_size (integer, Minimum: 0, Maximum: 268435456). The maximum request size in bytes.Appears on spec.
Logs configuration values.
Optional
elasticsearch_index_days_max (integer, Minimum: 1, Maximum: 10000). Elasticsearch index retention limit.elasticsearch_index_prefix (string, MinLength: 1, MaxLength: 1024). Elasticsearch index prefix.selected_log_fields (array of strings, MaxItems: 5). The list of logging fields that will be sent to the integration logging service. The MESSAGE and timestamp fields are always sent.Appears on spec.
Metrics configuration values.
Optional
database (string, Pattern: ^[_A-Za-z0-9][-_A-Za-z0-9]{0,39}$, MaxLength: 40). Name of the database where to store metric datapoints. Only affects PostgreSQL destinations. Defaults to metrics. Note that this must be the same for all metrics integrations that write data to the same PostgreSQL service.retention_days (integer, Minimum: 0, Maximum: 10000). Number of days to keep old metrics. Only affects PostgreSQL destinations. Set to 0 for no automatic cleanup. Defaults to 30 days.ro_username (string, Pattern: ^[_A-Za-z0-9][-._A-Za-z0-9]{0,39}$, MaxLength: 40). Name of a user that can be used to read metrics. This will be used for Grafana integration (if enabled) to prevent Grafana users from making undesired changes. Only affects PostgreSQL destinations. Defaults to metrics_reader. Note that this must be the same for all metrics integrations that write data to the same PostgreSQL service.source_mysql (object). Configuration options for metrics where source service is MySQL. See below for nested schema.username (string, Pattern: ^[_A-Za-z0-9][-._A-Za-z0-9]{0,39}$, MaxLength: 40). Name of the user used to write metrics. Only affects PostgreSQL destinations. Defaults to metrics_writer. Note that this must be the same for all metrics integrations that write data to the same PostgreSQL service.Appears on spec.metrics.
Configuration options for metrics where source service is MySQL.
Required
telegraf (object). Configuration options for Telegraf MySQL input plugin. See below for nested schema.Appears on spec.metrics.source_mysql.
Configuration options for Telegraf MySQL input plugin.
Optional
gather_event_waits (boolean). Gather metrics from PERFORMANCE_SCHEMA.EVENT_WAITS.gather_file_events_stats (boolean). gather metrics from PERFORMANCE_SCHEMA.FILE_SUMMARY_BY_EVENT_NAME.gather_index_io_waits (boolean). Gather metrics from PERFORMANCE_SCHEMA.TABLE_IO_WAITS_SUMMARY_BY_INDEX_USAGE.gather_info_schema_auto_inc (boolean). Gather auto_increment columns and max values from information schema.gather_innodb_metrics (boolean). Gather metrics from INFORMATION_SCHEMA.INNODB_METRICS.gather_perf_events_statements (boolean). Gather metrics from PERFORMANCE_SCHEMA.EVENTS_STATEMENTS_SUMMARY_BY_DIGEST.gather_process_list (boolean). Gather thread state counts from INFORMATION_SCHEMA.PROCESSLIST.gather_slave_status (boolean). Gather metrics from SHOW SLAVE STATUS command output.gather_table_io_waits (boolean). Gather metrics from PERFORMANCE_SCHEMA.TABLE_IO_WAITS_SUMMARY_BY_TABLE.gather_table_lock_waits (boolean). Gather metrics from PERFORMANCE_SCHEMA.TABLE_LOCK_WAITS.gather_table_schema (boolean). Gather metrics from INFORMATION_SCHEMA.TABLES.perf_events_statements_digest_text_limit (integer, Minimum: 1, Maximum: 2048). Truncates digest text from perf_events_statements into this many characters.perf_events_statements_limit (integer, Minimum: 1, Maximum: 4000). Limits metrics from perf_events_statements.perf_events_statements_time_limit (integer, Minimum: 1, Maximum: 2592000). Only include perf_events_statements whose last seen is less than this many seconds.apiVersion: aiven.io/v1alpha1\nkind: ServiceIntegrationEndpoint\nmetadata:\n name: my-service-integration-endpoint\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n project: aiven-project-name\n endpointName: my-external-postgresql\n endpointType: external_postgresql\n\n externalPostgresql:\n username: username\n password: password\n host: example.example\n port: 5432\n ssl_mode: require\napiVersion: aiven.io/v1alpha1\nkind: ServiceIntegrationEndpoint\nmetadata:\n name: my-service-integration-endpoint\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n project: aiven-project-name\n endpointName: my-external-schema-registry\n endpointType: external_schema_registry\n\n externalSchemaRegistry:\n url: https://schema-registry.example.com:8081\n authentication: basic\n basic_auth_username: username\n basic_auth_password: password\nInfo
To create this resource, a Secret containing Aiven token must be created first.
Apply the resource with:
kubectl apply -f example.yaml\nVerify the newly created ServiceIntegrationEndpoint:
kubectl get serviceintegrationendpoints my-service-integration-endpoint\nThe output is similar to the following:
Name Project Endpoint Name Endpoint Type ID \nmy-service-integration-endpoint aiven-project-name my-external-postgresql external_postgresql <id> \nServiceIntegrationEndpoint is the Schema for the serviceintegrationendpoints API.
Required
apiVersion (string). Value aiven.io/v1alpha1.kind (string). Value ServiceIntegrationEndpoint.metadata (object). Data that identifies the object, including a name string and optional namespace.spec (object). ServiceIntegrationEndpointSpec defines the desired state of ServiceIntegrationEndpoint. See below for nested schema.Appears on ServiceIntegrationEndpoint.
ServiceIntegrationEndpointSpec defines the desired state of ServiceIntegrationEndpoint.
Required
endpointType (string, Enum: autoscaler, datadog, external_aws_cloudwatch_logs, external_aws_cloudwatch_metrics, external_aws_s3, external_clickhouse, external_elasticsearch_logs, external_google_cloud_bigquery, external_google_cloud_logging, external_kafka, external_mysql, external_opensearch_logs, external_postgresql, external_redis, external_schema_registry, external_sumologic_logs, jolokia, prometheus, rsyslog, Immutable). Type of the service integration endpoint.project (string, Immutable, Pattern: ^[a-zA-Z0-9_-]+$, MaxLength: 63). Identifies the project this resource belongs to.Optional
authSecretRef (object). Authentication reference to Aiven token in a secret. See below for nested schema.datadog (object). Datadog configuration values. See below for nested schema.endpointName (string, Immutable, MaxLength: 36). Source endpoint for the integration (if any).externalAWSCloudwatchLogs (object). ExternalAwsCloudwatchLogs configuration values. See below for nested schema.externalAWSCloudwatchMetrics (object). ExternalAwsCloudwatchMetrics configuration values. See below for nested schema.externalElasticsearchLogs (object). ExternalElasticsearchLogs configuration values. See below for nested schema.externalGoogleCloudBigquery (object). ExternalGoogleCloudBigquery configuration values. See below for nested schema.externalGoogleCloudLogging (object). ExternalGoogleCloudLogging configuration values. See below for nested schema.externalKafka (object). ExternalKafka configuration values. See below for nested schema.externalOpensearchLogs (object). ExternalOpensearchLogs configuration values. See below for nested schema.externalPostgresql (object). ExternalPostgresql configuration values. See below for nested schema.externalSchemaRegistry (object). ExternalSchemaRegistry configuration values. See below for nested schema.jolokia (object). Jolokia configuration values. See below for nested schema.prometheus (object). Prometheus configuration values. See below for nested schema.rsyslog (object). Rsyslog configuration values. See below for nested schema.Appears on spec.
Authentication reference to Aiven token in a secret.
Required
key (string, MinLength: 1).name (string, MinLength: 1).Appears on spec.
Datadog configuration values.
Required
datadog_api_key (string, Pattern: ^[A-Za-z0-9]{1,256}$, MinLength: 1, MaxLength: 256). Datadog API key.Optional
datadog_tags (array of objects, MaxItems: 32). Custom tags provided by user. See below for nested schema.disable_consumer_stats (boolean). Disable consumer group metrics.kafka_consumer_check_instances (integer, Minimum: 1, Maximum: 100). Number of separate instances to fetch kafka consumer statistics with.kafka_consumer_stats_timeout (integer, Minimum: 2, Maximum: 300). Number of seconds that datadog will wait to get consumer statistics from brokers.max_partition_contexts (integer, Minimum: 200, Maximum: 200000). Maximum number of partition contexts to send.site (string, Enum: datadoghq.com, datadoghq.eu, us3.datadoghq.com, us5.datadoghq.com, ddog-gov.com, ap1.datadoghq.com). Datadog intake site. Defaults to datadoghq.com.Appears on spec.datadog.
Custom tags provided by user.
Required
tag (string, MinLength: 1, MaxLength: 200). Tag format and usage are described here: https://docs.datadoghq.com/getting_started/tagging. Tags with prefix aiven- are reserved for Aiven.Optional
comment (string, MaxLength: 1024). Optional tag explanation.Appears on spec.
ExternalAwsCloudwatchLogs configuration values.
Required
access_key (string, MaxLength: 4096). AWS access key. Required permissions are logs:CreateLogGroup, logs:CreateLogStream, logs:PutLogEvents and logs:DescribeLogStreams.region (string, MaxLength: 32). AWS region.secret_key (string, MaxLength: 4096). AWS secret key.Optional
log_group_name (string, Pattern: ^[\\.\\-_/#A-Za-z0-9]+$, MinLength: 1, MaxLength: 512). AWS CloudWatch log group name.Appears on spec.
ExternalAwsCloudwatchMetrics configuration values.
Required
access_key (string, MaxLength: 4096). AWS access key. Required permissions are cloudwatch:PutMetricData.namespace (string, MinLength: 1, MaxLength: 255). AWS CloudWatch Metrics Namespace.region (string, MaxLength: 32). AWS region.secret_key (string, MaxLength: 4096). AWS secret key.Appears on spec.
ExternalElasticsearchLogs configuration values.
Required
index_prefix (string, Pattern: ^[a-z0-9][a-z0-9-_.]+$, MinLength: 1, MaxLength: 1000). Elasticsearch index prefix.url (string, MinLength: 12, MaxLength: 2048). Elasticsearch connection URL.Optional
ca (string, MaxLength: 16384). PEM encoded CA certificate.index_days_max (integer, Minimum: 1, Maximum: 10000). Maximum number of days of logs to keep.timeout (number, Minimum: 10, Maximum: 120). Elasticsearch request timeout limit.Appears on spec.
ExternalGoogleCloudBigquery configuration values.
Required
project_id (string, MinLength: 6, MaxLength: 30). GCP project id.service_account_credentials (string, MaxLength: 4096). This is a JSON object with the fields documented in https://cloud.google.com/iam/docs/creating-managing-service-account-keys .Appears on spec.
ExternalGoogleCloudLogging configuration values.
Required
log_id (string, MaxLength: 512). Google Cloud Logging log id.project_id (string, MinLength: 6, MaxLength: 30). GCP project id.service_account_credentials (string, MaxLength: 4096). This is a JSON object with the fields documented in https://cloud.google.com/iam/docs/creating-managing-service-account-keys .Appears on spec.
ExternalKafka configuration values.
Required
bootstrap_servers (string, MinLength: 3, MaxLength: 256). Bootstrap servers.security_protocol (string, Enum: PLAINTEXT, SSL, SASL_PLAINTEXT, SASL_SSL). Security protocol.Optional
sasl_mechanism (string, Enum: PLAIN, SCRAM-SHA-256, SCRAM-SHA-512). SASL mechanism used for connections to the Kafka server.sasl_plain_password (string, MinLength: 1, MaxLength: 256). Password for SASL PLAIN mechanism in the Kafka server.sasl_plain_username (string, MinLength: 1, MaxLength: 256). Username for SASL PLAIN mechanism in the Kafka server.ssl_ca_cert (string, MaxLength: 16384). PEM-encoded CA certificate.ssl_client_cert (string, MaxLength: 16384). PEM-encoded client certificate.ssl_client_key (string, MaxLength: 16384). PEM-encoded client key.ssl_endpoint_identification_algorithm (string, Enum: https). The endpoint identification algorithm to validate server hostname using server certificate.Appears on spec.
ExternalOpensearchLogs configuration values.
Required
index_prefix (string, Pattern: ^[a-z0-9][a-z0-9-_.]+$, MinLength: 1, MaxLength: 1000). OpenSearch index prefix.url (string, MinLength: 12, MaxLength: 2048). OpenSearch connection URL.Optional
ca (string, MaxLength: 16384). PEM encoded CA certificate.index_days_max (integer, Minimum: 1, Maximum: 10000). Maximum number of days of logs to keep.timeout (number, Minimum: 10, Maximum: 120). OpenSearch request timeout limit.Appears on spec.
ExternalPostgresql configuration values.
Required
host (string, MaxLength: 255). Hostname or IP address of the server.port (integer, Minimum: 1, Maximum: 65535). Port number of the server.username (string, MaxLength: 256). User name.Optional
default_database (string, Pattern: ^[_A-Za-z0-9][-_A-Za-z0-9]{0,62}$, MaxLength: 63). Default database.password (string, MaxLength: 256). Password.ssl_client_certificate (string, MaxLength: 16384). Client certificate.ssl_client_key (string, MaxLength: 16384). Client key.ssl_mode (string, Enum: require, verify-ca, verify-full). SSL mode to use for the connection. Please note that Aiven requires TLS for all connections to external PostgreSQL services.ssl_root_cert (string, MaxLength: 16384). SSL Root Cert.Appears on spec.
ExternalSchemaRegistry configuration values.
Required
authentication (string, Enum: none, basic). Authentication method.url (string, MaxLength: 2048). Schema Registry URL.Optional
basic_auth_password (string, MaxLength: 256). Basic authentication password.basic_auth_username (string, MaxLength: 256). Basic authentication user name.Appears on spec.
Jolokia configuration values.
Optional
basic_auth_password (string, MinLength: 8, MaxLength: 64). Jolokia basic authentication password.basic_auth_username (string, Pattern: ^[a-z0-9\\-@_]{5,32}$, MinLength: 5, MaxLength: 32). Jolokia basic authentication username.Appears on spec.
Prometheus configuration values.
Optional
basic_auth_password (string, MinLength: 8, MaxLength: 64). Prometheus basic authentication password.basic_auth_username (string, Pattern: ^[a-z0-9\\-@_]{5,32}$, MinLength: 5, MaxLength: 32). Prometheus basic authentication username.Appears on spec.
Rsyslog configuration values.
Required
format (string, Enum: rfc5424, rfc3164, custom). Message format.port (integer, Minimum: 1, Maximum: 65535). Rsyslog server port.server (string, MinLength: 4, MaxLength: 255). Rsyslog server IP address or hostname.tls (boolean). Require TLS.Optional
ca (string, MaxLength: 16384). PEM encoded CA certificate.cert (string, MaxLength: 16384). PEM encoded client certificate.key (string, MaxLength: 16384). PEM encoded client key.logline (string, Pattern: ^[ -~\\t]+$, MinLength: 1, MaxLength: 512). Custom syslog message format.max_message_size (integer, Minimum: 2048, Maximum: 2147483647). Rsyslog max message size.sd (string, MaxLength: 1024). Structured data block for log message.apiVersion: aiven.io/v1alpha1\nkind: ServiceUser\nmetadata:\n name: my-service-user\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n connInfoSecretTarget:\n name: service-user-secret\n prefix: MY_SECRET_PREFIX_\n annotations:\n foo: bar\n labels:\n baz: egg\n\n project: aiven-project-name\n serviceName: my-service-name\nInfo
To create this resource, a Secret containing Aiven token must be created first.
Apply the resource with:
kubectl apply -f example.yaml\nVerify the newly created ServiceUser:
kubectl get serviceusers my-service-user\nThe output is similar to the following:
Name Service Name Project \nmy-service-user my-service-name aiven-project-name \nTo view the details of the Secret, use the following command:
kubectl describe secret service-user-secret\nYou can use the jq to quickly decode the Secret:
kubectl get secret service-user-secret -o json | jq '.data | map_values(@base64d)'\nThe output is similar to the following:
{\n \"SERVICEUSER_HOST\": \"<secret>\",\n \"SERVICEUSER_PORT\": \"<secret>\",\n \"SERVICEUSER_USERNAME\": \"<secret>\",\n \"SERVICEUSER_PASSWORD\": \"<secret>\",\n \"SERVICEUSER_CA_CERT\": \"<secret>\",\n \"SERVICEUSER_ACCESS_CERT\": \"<secret>\",\n \"SERVICEUSER_ACCESS_KEY\": \"<secret>\",\n}\nServiceUser is the Schema for the serviceusers API.
Exposes secret keys
SERVICEUSER_HOST, SERVICEUSER_PORT, SERVICEUSER_USERNAME, SERVICEUSER_PASSWORD, SERVICEUSER_CA_CERT, SERVICEUSER_ACCESS_CERT, SERVICEUSER_ACCESS_KEY.
Required
apiVersion (string). Value aiven.io/v1alpha1.kind (string). Value ServiceUser.metadata (object). Data that identifies the object, including a name string and optional namespace.spec (object). ServiceUserSpec defines the desired state of ServiceUser. See below for nested schema.Appears on ServiceUser.
ServiceUserSpec defines the desired state of ServiceUser.
Required
project (string, Immutable, Pattern: ^[a-zA-Z0-9_-]+$, MaxLength: 63). Identifies the project this resource belongs to.serviceName (string, Immutable, Pattern: ^[a-z][-a-z0-9]+$, MaxLength: 63). Specifies the name of the service that this resource belongs to.Optional
authSecretRef (object). Authentication reference to Aiven token in a secret. See below for nested schema.authentication (string, Enum: caching_sha2_password, mysql_native_password). Authentication details.connInfoSecretTarget (object). Secret configuration. See below for nested schema.connInfoSecretTargetDisabled (boolean, Immutable). When true, the secret containing connection information will not be created, defaults to false. This field cannot be changed after resource creation.Appears on spec.
Authentication reference to Aiven token in a secret.
Required
key (string, MinLength: 1).name (string, MinLength: 1).Appears on spec.
Secret configuration.
Required
name (string, Immutable). Name of the secret resource to be created. By default, it is equal to the resource name.Optional
annotations (object, AdditionalProperties: string). Annotations added to the secret.labels (object, AdditionalProperties: string). Labels added to the secret.prefix (string). Prefix for the secret's keys. Added \"as is\" without any transformations. By default, is equal to the kind name in uppercase + underscore, e.g. KAFKA_, REDIS_, etc.The Aiven Operator for Kubernetes project accepts contributions via GitHub pull requests. This document outlines the process to help get your contribution accepted.
Please see also the Aiven Operator for Kubernetes Developer Guide.
"},{"location":"contributing/index.html#support-channels","title":"Support Channels","text":"This project offers support through GitHub issues and can be filed here. Moreover, GitHub issues are used as the primary method for tracking anything to do with the Aiven Operator for Kubernetes project.
"},{"location":"contributing/index.html#pull-request-process","title":"Pull Request Process","text":"In the interest of fostering an open and welcoming environment, we as contributors and maintainers pledge to making participation in our project and our community a harassment-free experience for everyone, regardless of age, body size, disability, ethnicity, gender identity and expression, level of experience, nationality, personal appearance, race, religion, or sexual identity and orientation.
"},{"location":"contributing/index.html#our-standards","title":"Our Standards","text":"Examples of behavior that contributes to creating a positive environment include:
Examples of unacceptable behavior by participants include:
This project adheres to the Conventional Commits specification. Please, make sure that your commit messages follow that specification.
"},{"location":"contributing/index.html#our-responsibilities","title":"Our Responsibilities","text":"Project maintainers are responsible for clarifying the standards of acceptable behavior and are expected to take appropriate and fair corrective action in response to any instances of unacceptable behavior.
Project maintainers have the right and responsibility to remove, edit, or reject comments, commits, code, wiki edits, issues, and other contributions that are not aligned to this Code of Conduct, or to ban temporarily or permanently any contributor for other behaviors that they deem inappropriate, threatening, offensive, or harmful.
"},{"location":"contributing/index.html#scope","title":"Scope","text":"This Code of Conduct applies both within project spaces and in public spaces when an individual is representing the project or its community. Examples of representing a project or community include using an official project e-mail address, posting via an official social media account, or acting as an appointed representative at an online or offline event. Representation of a project may be further defined and clarified by project maintainers.
"},{"location":"contributing/index.html#enforcement","title":"Enforcement","text":"Instances of abusive, harassing, or otherwise unacceptable behavior may be reported by contacting the project team at opensource@aiven.io. All complaints will be reviewed and investigated and will result in a response that is deemed necessary and appropriate to the circumstances. The project team is obligated to maintain confidentiality with regard to the reporter of an incident. Further details of specific enforcement policies may be posted separately.
Project maintainers who do not follow or enforce the Code of Conduct in good faith may face temporary or permanent repercussions as determined by other members of the project's leadership.
"},{"location":"contributing/developer-guide.html","title":"Developer guide","text":""},{"location":"contributing/developer-guide.html#get-started","title":"Get Started","text":"To get started, make sure you have a working Go environment and clone the repository:
git clone git@github.com:aiven/aiven-operator.git\ncd aiven-operator\nBefore proceeding, please ensure that you have the following dependencies installed:
-w0 flag, which may cause some tests to not work properly)Create an Aiven authentication token and an Aiven project, you'll need them in the following sections:
AIVEN_TOKEN \u2014 your authentication tokenAIVEN_PROJECT_NAME \u2014 the name of you Aiven project to run the services inOptional dependencies:
The project uses the make build system. To see the available targets, run make help.
Building the operator binary:
make build\nTo run the operator locally, first create a local k8s cluster with kind:
$ kind create cluster --image kindest/node:v1.26.6 --wait 5m\nCreating cluster \"kind\" ...\n \u2713 Ensuring node image (kindest/node:v1.26.6) \ud83d\uddbc\n \u2713 Preparing nodes \ud83d\udce6\n \u2713 Writing configuration \ud83d\udcdc\n \u2713 Starting control-plane \ud83d\udd79\ufe0f\n \u2713 Installing CNI \ud83d\udd0c\n \u2713 Installing StorageClass \ud83d\udcbe\n \u2713 Waiting \u2264 5m0s for control-plane = Ready \u23f3\n \u2022 Ready after 14s \ud83d\udc9a\nSet kubectl context to \"kind-kind\"\nYou can now use your cluster with:\n\nkubectl cluster-info --context kind-kinds\nThis creates a k8s cluster and points your kubectl to that cluster:
$ kubectl cluster-info --context kind-kind\nKubernetes control plane is running at https://127.0.0.1:58083\nCoreDNS is running at https://127.0.0.1:58083/api/v1/namespaces/kube-system/services/kube-dns:dns/proxy\nRun the operator:
ENABLE_WEBHOOKS=false make run\nIn another terminal window, create a Kubernetes Secret named aiven-token (use the Aiven authentication token created earlier):
export AIVEN_TOKEN=...\nkubectl create secret generic aiven-token --from-literal=token=$AIVEN_TOKEN\nThen, create a manifest for an Aiven PostgreSQL service:
aiven-pg.yamlapiVersion: aiven.io/v1alpha1\nkind: PostgreSQL\nmetadata:\n name: aiven-devdocs-pg\nspec:\n # reads the authentication token\n authSecretRef:\n name: aiven-token\n key: token\n\n # stores the PostgreSQL connection information on the specified Secret\n connInfoSecretTarget:\n name: pg-connection\n\n project: <your-project-name>\n cloudName: google-europe-west1\n plan: hobbyist\n maintenanceWindowDow: friday\n maintenanceWindowTime: 23:00:00\n userConfig:\n pg_version: \"15\"\n...and create the service at Aiven by applying the manifest:
kubectl apply -f aiven-pg.yaml\nYou should see the service be created on Aiven's side using either Aiven Console or Aiven CLI:
$ avn project switch PROJECT_NAME\n$ avn service list\nSERVICE_NAME SERVICE_TYPE STATE CLOUD_NAME PLAN CREATE_TIME UPDATE_TIME NOTIFICATIONS\n================ ============ ========== =================== ======== ==================== ==================== =============\naiven-devdocs-pg pg REBUILDING google-europe-west1 hobbyist 2024-01-01T00:00:00Z 2024-01-01T00:01:00Z\nTo see the resource on k8s side, use kubectl:
$ kubectl get postgresqls\nNAME PROJECT REGION PLAN STATE\naiven-devdocs-pg aiven-sbox-k8s-riski google-europe-west1 hobbyist REBUILDING\n... or by running k9s in another terminal window and type \": postgresqls\" to see the provisioned PostgreSQL resources:
For this section you'll need to set export AIVEN_TOKEN=... and an export AIVEN_PROJECT_NAME=... as well as export ENABLE_WEBHOOKS=false (see Prerequisites).
Note
If you encounter any issues with webhooks, you have the option to disable them by running export ENABLE_WEBOOKS=false
Running all unit tests is possible by running make test. To run a targeted test, use run=TestGrafana make test.
Setup the test environment:
make e2e-setup-kind\nThe e2e tests are configured to run against a project called aiven-ci-kubernetes-operator. To run the test against
make test-e2e-preinstalled\nTo remove the kind cluster, simply execute the following command:
kind delete cluster\nPlease see this page for more information.
"},{"location":"contributing/developer-guide.html#documentation","title":"Documentation","text":"The documentation is written in markdown and generated by mkdocs and mkdocs-material.
To run the documentation live preview:
make serve-docs\nAnd open the http://localhost:8000/aiven-operator/ page in your web browser.
The documentation API Reference section is generated automatically from the source code during the documentation deployment. To generate it locally, run the following command:
make docs\nAiven Kubernetes Operator generates service configs code (also known as user configs) and documentation from public service types schema.
"},{"location":"contributing/resource-generation.html#flow-overview","title":"Flow overview","text":"When a new schema is issued on the API, a cron job fetches it, parses, patches, and saves in a shared library \u2014 go-api-schemas.
When the library is updated, the GitHub dependabot creates PRs to the dependent repositories, like Aiven Kubernetes Operator and Aiven Terraform Provider.
Then the make generate command is called by GitHub action. And the PR is ready for review.
flowchart TB\n API(Aiven API) <-.->|polls schema updates| Schema([go-api-schemas])\n Bot(dependabot) <-.->|polls updates| Schema\n Bot-->|pull request|UpdateOP[/\"\u2728 $ make generate \u2728\"/]\n UpdateOP-->|review| OP([operator repository])The command runs several generators in a certain sequence. First, the user config generator is called. Then controller-gen cli. Then API reference docs generator and charts generator.
Here how it goes in the details:
./<api-reference-docs>/example/, if it finds one, it validates that with the CRD. Each CRD has an OpenAPI v3 schema as a part of it. This is also used by Kubernetes itself to validate user input.flowchart TB\n Make[/$ make generate/]-->Generator(userconfig generator<br> creates/updates structs using updated spec)\n Generator-->|go: KafkaUserConfig struct| K8S(controller-gen<br> adds k8s methods to structs)\n K8S-->|go files| CRD(controller-gen<br> creates CRDs out of structs)\n CRD-->|CRD: aiven.io_kafkas.yaml| Docs(docs generator)\n subgraph API reference generation\n Docs-->|aiven.io_kafkas.yaml|Reference(creates reference<br> out of CRD)\n Docs-->|examples/kafka.yaml,<br> aiven.io_kafkas.yaml|Examples(validates example<br> using CRD)\n Examples--> Markdown(creates docs out of CRDs, adds examples)\n Reference-->Markdown(kafka.md)\n end\n CRD-->|yaml files|Charts(charts generator<br> updates helm charts<br> and the changelog)\n Charts-->ToRelease(\"Ready to release \ud83c\udf89\")\n Markdown-->ToReleaseBy default, charts generator keeps the current helm chart's version, because it doesn't know semver. You need it to do manually.
To do so run the following command with the version of your choice:
make version=v1.0.0 charts\nThe Aiven Operator for Kubernetes can be installed via Helm.
Before you start, make sure you have the prerequisites.
First add the Aiven Helm repository:
helm repo add aiven https://aiven.github.io/aiven-charts && helm repo update\nhelm install aiven-operator-crds aiven/aiven-operator-crds\nVerify the installation:
kubectl api-resources --api-group=aiven.io\nThe output is similar to the following:
NAME SHORTNAMES APIVERSION NAMESPACED KIND\nconnectionpools aiven.io/v1alpha1 true ConnectionPool\ndatabases aiven.io/v1alpha1 true Database\n... < several omitted lines >\nhelm install aiven-operator aiven/aiven-operator\nNote
Installation will fail if webhooks are enabled and the CRDs for the cert-manager are not installed.
Verify the installation:
helm status aiven-operator\nThe output is similar to the following:
NAME: aiven-operator\nLAST DEPLOYED: Fri Sep 10 15:23:26 2021\nNAMESPACE: default\nSTATUS: deployed\nREVISION: 1\nTEST SUITE: None\nIt is also possible to install the operator without webhooks enabled:
helm install aiven-operator aiven/aiven-operator --set webhooks.enabled=false\nPlease refer to the values.yaml of the chart.
"},{"location":"installation/helm.html#install-without-full-cluster-administrator-access","title":"Install without full cluster administrator access","text":"If the person installing the Helm chart does not have the necessary permissions to create cluster-wide resources such as ClusterRole and ClusterRoleBinding, a cluster administrator can manually install these roles. This ensures that the operator can function properly.
Important
Please see this page for more information.
Find out the name of your deployment:
helm list\nThe output has the name of each deployment similar to the following:
NAME NAMESPACE REVISION UPDATED STATUS CHART APP VERSION\naiven-operator default 1 2021-09-09 10:56:14.623700249 +0200 CEST deployed aiven-operator-v0.1.0 v0.1.0\naiven-operator-crds default 1 2021-09-09 10:56:05.736411868 +0200 CEST deployed aiven-operator-crds-v0.1.0 v0.1.0\nRemove the CRDs:
helm uninstall aiven-operator-crds\nThe confirmation message is similar to the following:
release \"aiven-operator-crds\" uninstalled\nRemove the operator:
helm uninstall aiven-operator\nThe confirmation message is similar to the following:
release \"aiven-operator\" uninstalled\nBefore you start, make sure you have the prerequisites.
All Aiven Operator for Kubernetes components can be installed from one YAML file that is uploaded for every release:
kubectl apply -f https://github.com/aiven/aiven-operator/releases/latest/download/deployment.yaml\nBy default the Deployment is installed into the aiven-operator-system namespace.
Assuming you installed version vX.Y.Z of the operator it can be uninstalled via
kubectl delete -f https://github.com/aiven/aiven-operator/releases/download/vX.Y.Z/deployment.yaml\nThe Aiven Operator for Kubernetes supports all major Kubernetes distributions, both locally and in the cloud.
Make sure you have the following:
The Aiven Operator for Kubernetes uses cert-manager to configure the service reference of our webhooks.
Please follow the installation instructions on their website.
Note
This is not required in the Helm installation if you select to disable webhooks, but that is not recommended outside of playground use.
The Aiven Operator for Kubernetes uses webhooks for setting defaults and enforcing invariants that are expected by the aiven API and will lead to errors if ignored.
In the future webhooks will also be used for conversion and supporting multiple CRD versions.
"},{"location":"installation/uninstalling.html","title":"Uninstall","text":"Danger
Uninstalling the Aiven Operator for Kubernetes can remove the resources created in Aiven, possibly resulting in data loss.
Depending on your installation, please follow one of:
Aiven resources need to have an accompanying secret that contains the token that is used to authorize the manipulation of that resource. If that token expired then you will not be able to delete the custom resource and deletion will also hang until the situation is resolved. The recommended approach to deal with that situation is to patch a valid token into the secret again so that proper cleanup of aiven resources can take place.
"},{"location":"installation/uninstalling.html#hanging-deletions","title":"Hanging deletions","text":"To protect the Secrets that the operator is using from deletion, it adds the finalizer finalizers.aiven.io/needed-to-delete-services to the Secret. This solves a race condition that happens when deleting a namespace, where there is a possibility of the Secret getting deleted before the resource that uses it. When the controller is deleted it may not cleanup the finalizers from all Secrets. If there is a Secret with this finalizer blocking deletion of a namespace, you can remove the finalizer by running:
kubectl patch secret <offending-secret> -p '{\"metadata\":{\"finalizers\":null}}' --type=merge\nAiven for Apache Cassandra\u00ae is a distributed database designed to handle large volumes of writes.
Note
Before going through this guide, make sure you have a Kubernetes cluster with the operator installed (see instructions for helm or kubectl), and a Kubernetes Secret with an Aiven authentication token.
"},{"location":"resources/cassandra.html#create-a-cassandra-instance","title":"Create a Cassandra instance","text":"1. Create a file named cassandra-sample.yaml, and add the following content:
apiVersion: aiven.io/v1alpha1\nkind: Cassandra\nmetadata:\n name: cassandra-sample\nspec:\n # gets the authentication token from the `aiven-token` Secret\n authSecretRef:\n name: aiven-token\n key: token\n\n # outputs the Cassandra connection on the `cassandra-secret` Secret\n connInfoSecretTarget:\n name: cassandra-secret\n\n # add your Project name here\n project: PROJECT_NAME\n\n # cloud provider and plan of your choice\n # you can check all of the possibilities here https://aiven.io/pricing\n cloudName: google-europe-west1\n plan: startup-4\n\n # general Aiven configuration\n maintenanceWindowDow: friday\n maintenanceWindowTime: 23:00:00\n2. Create the service by applying the configuration:
kubectl apply -f cassandra-sample.yaml\nThe output is:
cassandra.aiven.io/cassandra-sample created\n3. Review the resource you created with this command:
kubectl describe cassandra.aiven.io cassandra-sample\nThe output is similar to the following:
...\nStatus:\n Conditions:\n Last Transition Time: 2023-01-31T10:17:25Z\n Message: Successfully created or updated the instance in Aiven\n Reason: Created\n Status: True\n Type: Initialized\n Last Transition Time: 2023-01-31T10:24:00Z\n Message: Instance is running on Aiven side\n Reason: CheckRunning\n Status: True\n Type: Running\n State: RUNNING\n...\nThe resource can be in the REBUILDING state for a few minutes. Once the state changes to RUNNING, you can access the resource.
For your convenience, the operator automatically stores the Cassandra connection information in a Secret created with the name specified on the connInfoSecretTarget field.
To view the details of the Secret, use the following command:
kubectl describe secret cassandra-secret\nThe output is similar to the following:
Name: cassandra-secret\nNamespace: default\nLabels: <none>\nAnnotations: <none>\n\nType: Opaque\n\nData\n====\nCASSANDRA_HOSTS: 59 bytes\nCASSANDRA_PASSWORD: 24 bytes\nCASSANDRA_PORT: 5 bytes\nCASSANDRA_URI: 66 bytes\nCASSANDRA_USER: 8 bytes\nCASSANDRA_HOST: 60 bytes\nYou can use the jq to quickly decode the Secret:
kubectl get secret cassandra-secret -o json | jq '.data | map_values(@base64d)'\nThe output is similar to the following:
{\n \"CASSANDRA_HOST\": \"<secret>\",\n \"CASSANDRA_HOSTS\": \"<secret>\",\n \"CASSANDRA_PASSWORD\": \"<secret>\",\n \"CASSANDRA_PORT\": \"14609\",\n \"CASSANDRA_URI\": \"<secret>\",\n \"CASSANDRA_USER\": \"avnadmin\"\n}\nYou can create service users for your instance of Aiven for Apache Cassandra. Service users are unique to this instance and are not shared with any other services.
1. Create a file named cassandra-service-user.yaml:
apiVersion: aiven.io/v1alpha1\nkind: ServiceUser\nmetadata:\n name: cassandra-service-user\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n connInfoSecretTarget:\n name: cassandra-service-user-secret\n\n project: PROJECT_NAME\n serviceName: cassandra-sample\n2. Create the user by applying the configuration:
kubectl apply -f cassandra-service-user.yaml\nThe ServiceUser resource generates a Secret with connection information.
3. View the details of the Secret using the following command:
kubectl get secret cassandra-service-user-secret -o json | jq '.data | map_values(@base64d)'\nThe output is similar to the following:
{\n \"ACCESS_CERT\": \"<secret>\",\n \"ACCESS_KEY\": \"<secret>\",\n \"CA_CERT\": \"<secret>\",\n \"HOST\": \"<secret>\",\n \"PASSWORD\": \"<secret>\",\n \"PORT\": \"14609\",\n \"USERNAME\": \"cassandra-service-user\"\n}\nYou can connect to the Cassandra instance using these credentials and the host information from the cassandra-secret Secret.
Aiven for ClickHouse\u00ae is a fully managed distributed columnar database based on open source ClickHouse.
Note
Before going through this guide, make sure you have a Kubernetes cluster with the operator installed (see instructions for helm or kubectl), and a Kubernetes Secret with an Aiven authentication token.
"},{"location":"resources/clickhouse.html#create-a-clickhouse-instance","title":"Create a ClickHouse instance","text":"1. Create a file named clickhouse-sample.yaml, and add the following:
apiVersion: aiven.io/v1alpha1\nkind: Clickhouse\nmetadata:\n name: example-clickhouse\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n # Outputs the ClickHouse connection to the `clickhouse-secret` Secret.\n connInfoSecretTarget:\n name: clickhouse-secret\n # Your Aiven project.\n project: PROJECT_NAME\n # Choose a cloud provider and plan.\n # View the options on the pricing page at https://aiven.io/pricing.\n cloudName: google-europe-west1\n plan: startup-16\n # Configure the maintenance window.\n maintenanceWindowDow: friday\n maintenanceWindowTime: 23:00:00\nPROJECT_NAME is the name of your Aiven project. 2. To create the service, run:
kubectl apply -f clickhouse-sample.yaml\nThe output is similar to the following:
clickhouse.aiven.io/example-clickhouse created\n3. To review the resource you created, run:
kubectl describe clickhouse.aiven.io example-clickhouse\nThe output is similar to the following:
...\nStatus:\n Conditions:\n Last Transition Time: 2024-06-25T07:58:37Z\n Message: Instance was created or update on Aiven side\n Reason: Created\n Status: True\n Type: Initialized\n Last Transition Time: 2024-06-25T08:01:47Z\n Message: Instance is running on Aiven side\n Reason: CheckRunning\n Status: True\n Type: Running\n State: RUNNING\nEvents:\n Type Reason Age From Message\n ---- ------ ---- ---- -------\n Normal CreatedOrUpdatedAtAiven 3m24s (x5 over 3m45s) clickhouse-reconciler waiting for the instance to be running\n Normal ReconcilationStarted 3m14s (x6 over 3m47s) clickhouse-reconciler starting reconciliation\n Normal InstanceFinalizerAdded 3m14s (x6 over 3m47s) clickhouse-reconciler waiting for preconditions of the instance\n...\nREBUILDING state for a few minutes. Once the state changes to RUNNING, you can access the service."},{"location":"resources/clickhouse.html#use-the-connection-secret","title":"Use the connection Secret","text":"Aiven Operator automatically stores the ClickHouse connection information in a Secret created with the name in the connInfoSecretTarget field.
To view the details of the Secret, run:
kubectl describe secret clickhouse-secret\nName: clickhouse-secret\nNamespace: default\nLabels: <none>\nAnnotations: <none>\n\nType: Opaque\n\nData\n====\nCLICKHOUSE_USER: 8 bytes\nHOST: 46 bytes\nPASSWORD: 24 bytes\nPORT: 5 bytes\nUSER: 8 bytes\nCLICKHOUSE_HOST: 46 bytes\nCLICKHOUSE_PASSWORD: 24 bytes\nCLICKHOUSE_PORT: 5 bytes\nYou can use the JSON processor jq to decode the Secret:
kubectl get secret clickhouse-secret -o json | jq '.data | map_values(@base64d)'\nThe output is similar to the following:
{\n \"CLICKHOUSE_HOST\": \"HOST\",\n \"CLICKHOUSE_PASSWORD\": \"SERVICE_PASSWORD\",\n \"CLICKHOUSE_PORT\": \"12691\",\n \"CLICKHOUSE_USER\": \"avnadmin\",\n \"HOST\": \"HOST\",\n \"PASSWORD\": \"ADMIN_USER_PASSWORD\",\n \"PORT\": \"12691\",\n \"USER\": \"avnadmin\"\n}\nNote
Tables cannot be created using Aiven Operator. To create a table, use the Aiven Console or CLI.
1. Create a file named clickhouse-db.yaml and add the following:
apiVersion: aiven.io/v1alpha1\nkind: ClickhouseDatabase\nmetadata:\n name: example-database\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n serviceName: example-clickhouse\n project: PROJECT_NAME\nWhere PROJECT_NAME is the name of your Aiven project.
2. To create the database, run:
kubectl apply -f clickhouse-db.yaml\n3. To view details about the resource, run:
kubectl describe ClickhouseDatabase example-database\nThe output is similar to the following:
...\nSpec:\n Auth Secret Ref:\n Key: token\n Name: aiven-token\n Project: example-project\n Service Name: example-clickhouse\nStatus:\n Conditions:\n Last Transition Time: 2024-06-25T13:58:35Z\n Message: Checking preconditions\n Reason: Preconditions\n Status: True\n Type: Initialized\n Last Transition Time: 2024-06-25T14:32:05Z\n Message: Instance is running on Aiven side\n Reason: CheckRunning\n Status: True\n Type: Running\n...\nYou can create service users and roles for an instance of ClickHouse, and grant privileges to them. Users and roles are not shared with any other services.
1. Create a file named clickhouse-service-users.yaml and add the following configuration for a service user:
apiVersion: aiven.io/v1alpha1\nkind: ClickhouseUser\nmetadata:\n name: example-user\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n connInfoSecretTarget:\n name: clickhouse-service-user-secret\n serviceName: example-clickhouse\n project: PROJECT_NAME\nPROJECT_NAME is the name of your Aiven project. This resource generates a Secret with connection information and stores it in clickhouse-service-user-secret.
2. To create a role add the following to the same file:
---\n\napiVersion: aiven.io/v1alpha1\nkind: ClickhouseRole\nmetadata:\n name: example-role\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n serviceName: example-clickhouse\n project: PROJECT_NAME\n role: read-only\n3. Grant privileges to the role, and assign the role to the user by adding the following to the same file:
---\n\napiVersion: aiven.io/v1alpha1\nkind: ClickhouseGrant\nmetadata:\n name: example-grant\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n project: PROJECT_NAME\n serviceName: example-clickhouse\n\n privilegeGrants:\n - grantees:\n - role: read-only\n privileges:\n - SELECT\n database: example-database\n\n roleGrants:\n - roles:\n - read-only\n grantees:\n - user: example-user\n4. To create the user, role, and grant, run:
kubectl apply -f clickhouse-service-users.yaml\n5. To get credentials and host information for connecting to this instance, view the connection information stored in the Secret by running:
kubectl get secret clickhouse-service-user-secret -o json | jq '.data | map_values(@base64d)'\nThe output is similar to the following:
{\n \"CLICKHOUSEUSER_HOST\": \"HOST\",\n \"CLICKHOUSEUSER_PASSWORD\": \"PASSWORD\",\n \"CLICKHOUSEUSER_PORT\": \"12691\",\n \"CLICKHOUSEUSER_USERNAME\": \"example-user\",\n \"HOST\": \"HOST\",\n \"PASSWORD\": \"PASSWORD\",\n \"PORT\": \"12691\",\n \"USERNAME\": \"example-user\"\n}\nAiven for MySQL is a fully managed relational database service, deployable in the cloud of your choice.
Before going through this guide, make sure you have a Kubernetes cluster with the operator installed (see instructions for helm or kubectl) and a Kubernetes Secret with an Aiven authentication token.
"},{"location":"resources/mysql.html#create-a-mysql-instance","title":"Create a MySQL instance","text":"1. Create a file named mysql-sample.yaml, and add the following content:
apiVersion: aiven.io/v1alpha1\nkind: MySQL\nmetadata:\n name: mysql-sample\nspec:\n # gets the authentication token from the `aiven-token` Secret\n authSecretRef:\n name: aiven-token\n key: token\n\n # outputs the MySQL connection on the `mysql-secret` Secret\n connInfoSecretTarget:\n name: mysql-secret\n\n # add your Project name here\n project: PROJECT_NAME\n\n # cloud provider and plan of your choice\n # you can check all of the possibilities here https://aiven.io/pricing\n cloudName: google-europe-west1\n plan: startup-4\n\n # general Aiven configuration\n maintenanceWindowDow: friday\n maintenanceWindowTime: 23:00:00\n2. Create the service by applying the configuration:
kubectl apply -f mysql-sample.yaml\n3. Review the resource you created with this command:
kubectl describe mysql.aiven.io mysql-sample\nThe output is similar to the following:
...\nStatus:\n Conditions:\n Last Transition Time: 2023-02-22T15:43:44Z\n Message: Successfully created or updated the instance in Aiven\n Reason: Created\n Status: True\n Type: Initialized\n Last Transition Time: 2023-02-22T15:43:44Z\n Message: Successfully created or updated the instance in Aiven, status remains unknown\n Reason: Created\n Status: Unknown\n Type: Running\n State: REBUILDING\n...\nThe resource will be in the REBUILDING state for a few minutes. Once the state changes to RUNNING, you can access the resource.
For your convenience, the operator automatically stores the MySQL connection information in a Secret created with the name specified on the connInfoSecretTarget field.
To view the details of the Secret, use the following command:
kubectl describe secret mysql-secret\nThe output is similar to the following:
Name: mysql-secret\nNamespace: default\nLabels: <none>\nAnnotations: <none>\n\nType: Opaque\n\nData\n====\nMYSQL_PORT: 5 bytes\nMYSQL_SSL_MODE: 8 bytes\nMYSQL_URI: 115 bytes\nMYSQL_USER: 8 bytes\nMYSQL_DATABASE: 9 bytes\nMYSQL_HOST: 39 bytes\nMYSQL_PASSWORD: 24 bytes\nYou can use jq to quickly decode the Secret:
kubectl get secret mysql-secret -o json | jq '.data | map_values(@base64d)'\nThe output is similar to the following:
{\n \"MYSQL_DATABASE\": \"defaultdb\",\n \"MYSQL_HOST\": \"<secret>\",\n \"MYSQL_PASSWORD\": \"<secret>\",\n \"MYSQL_PORT\": \"12691\",\n \"MYSQL_SSL_MODE\": \"REQUIRED\",\n \"MYSQL_URI\": \"<secret>\",\n \"MYSQL_USER\": \"avnadmin\"\n}\nYou can create service users for your instance of Aiven for MySQL. Service users are unique to this instance and are not shared with any other services.
1. Create a file named mysql-service-user.yaml:
apiVersion: aiven.io/v1alpha1\nkind: ServiceUser\nmetadata:\n name: mysql-service-user\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n connInfoSecretTarget:\n name: mysql-service-user-secret\n\n project: PROJECT_NAME\n serviceName: mysql-sample\n2. Create the user by applying the configuration:
kubectl apply -f mysql-service-user.yaml\nThe ServiceUser resource generates a Secret with connection information.
3. View the details of the Secret using jq:
kubectl get secret mysql-service-user-secret -o json | jq '.data | map_values(@base64d)'\nThe output is similar to the following:
{\n \"ACCESS_CERT\": \"<secret>\",\n \"ACCESS_KEY\": \"<secret>\",\n \"CA_CERT\": \"<secret>\",\n \"HOST\": \"<secret>\",\n \"PASSWORD\": \"<secret>\",\n \"PORT\": \"14609\",\n \"USERNAME\": \"mysql-service-user\"\n}\nYou can connect to the MySQL instance using these credentials and the host information from the mysql-secret Secret.
OpenSearch\u00ae is an open source search and analytics suite including search engine, NoSQL document database, and visualization interface. OpenSearch offers a distributed, full-text search engine based on Apache Lucene\u00ae with a RESTful API interface and support for JSON documents.
Note
Before going through this guide, make sure you have a Kubernetes cluster with the operator installed (see instructions for helm or kubectl), and a Kubernetes Secret with an Aiven authentication token.
"},{"location":"resources/opensearch.html#create-an-opensearch-instance","title":"Create an OpenSearch instance","text":"1. Create a file named os-sample.yaml, and add the following content:
apiVersion: aiven.io/v1alpha1\nkind: OpenSearch\nmetadata:\n name: os-sample\nspec:\n # gets the authentication token from the `aiven-token` Secret\n authSecretRef:\n name: aiven-token\n key: token\n\n # outputs the OpenSearch connection on the `os-secret` Secret\n connInfoSecretTarget:\n name: os-secret\n\n # add your Project name here\n project: PROJECT_NAME\n\n # cloud provider and plan of your choice\n # you can check all of the possibilities here https://aiven.io/pricing\n cloudName: google-europe-west1\n plan: startup-4\n\n # general Aiven configuration\n maintenanceWindowDow: friday\n maintenanceWindowTime: 23:00:00\n2. Create the service by applying the configuration:
kubectl apply -f os-sample.yaml\n3. Review the resource you created with this command:
kubectl describe opensearch.aiven.io os-sample\nThe output is similar to the following:
...\nStatus:\n Conditions:\n Last Transition Time: 2023-01-19T14:41:43Z\n Message: Successfully created or updated the instance in Aiven\n Reason: Created\n Status: True\n Type: Initialized\n Last Transition Time: 2023-01-19T14:41:43Z\n Message: Successfully created or updated the instance in Aiven, status remains unknown\n Reason: Created\n Status: Unknown\n Type: Running\n State: REBUILDING\n...\nThe resource will be in the REBUILDING state for a few minutes. Once the state changes to RUNNING, you can access the resource.
For your convenience, the operator automatically stores the OpenSearch connection information in a Secret created with the name specified on the connInfoSecretTarget field.
To view the details of the Secret, use the following command:
kubectl describe secret os-secret\nThe output is similar to the following:
Name: os-secret\nNamespace: default\nLabels: <none>\nAnnotations: <none>\n\nType: Opaque\n\nData\n====\nHOST: 61 bytes\nPASSWORD: 24 bytes\nPORT: 5 bytes\nUSER: 8 bytes\nYou can use the jq to quickly decode the Secret:
kubectl get secret os-secret -o json | jq '.data | map_values(@base64d)'\nThe output is similar to the following:
{\n \"HOST\": \"os-sample-your-project.aivencloud.com\",\n \"PASSWORD\": \"<secret>\",\n \"PORT\": \"13041\",\n \"USER\": \"avnadmin\"\n}\nYou can create service users for your instance of Aiven for OpenSearch. Service users are unique to this instance and are not shared with any other services.
1. Create a file named os-service-user.yaml:
apiVersion: aiven.io/v1alpha1\nkind: ServiceUser\nmetadata:\n name: os-service-user\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n connInfoSecretTarget:\n name: os-service-user-secret\n\n project: PROJECT_NAME\n serviceName: os-sample\n2. Create the user by applying the configuration:
kubectl apply -f os-service-user.yaml\nThe ServiceUser resource generates a Secret with connection information.
3. View the details of the Secret using the following command:
kubectl get secret os-service-user-secret -o json | jq '.data | map_values(@base64d)'\nThe output is similar to the following:
{\n \"ACCESS_CERT\": \"<secret>\",\n \"ACCESS_KEY\": \"<secret>\",\n \"CA_CERT\": \"<secret>\",\n \"HOST\": \"os-sample-your-project.aivencloud.com\",\n \"PASSWORD\": \"<secret>\",\n \"PORT\": \"14609\",\n \"USERNAME\": \"os-service-user\"\n}\nYou can connect to the OpenSearch instance using these credentials and the host information from the os-secret Secret.
PostgreSQL is an open source, relational database. It's ideal for organisations that need a well organised tabular datastore. On top of the strict table and columns formats, PostgreSQL also offers solutions for nested datasets with the native jsonb format and advanced set of extensions including PostGIS, a spatial database extender for location queries. Aiven for PostgreSQL is the perfect fit for your relational data.
With Aiven Kubernetes Operator, you can manage Aiven for PostgreSQL through the well defined Kubernetes API.
Note
Before going through this guide, make sure you have a Kubernetes cluster with the operator installed (see instructions for helm or kubectl), and a Kubernetes Secret with an Aiven authentication token.
"},{"location":"resources/postgresql.html#create-a-postgresql-instance","title":"Create a PostgreSQL instance","text":"1. Create a file named pg-sample.yaml with the following content:
apiVersion: aiven.io/v1alpha1\nkind: PostgreSQL\nmetadata:\n name: pg-sample\nspec:\n # gets the authentication token from the `aiven-token` Secret\n authSecretRef:\n name: aiven-token\n key: token\n\n # outputs the PostgreSQL connection on the `pg-connection` Secret\n connInfoSecretTarget:\n name: pg-connection\n\n # add your Project name here\n project: PROJECT_NAME\n\n # cloud provider and plan of your choice\n # you can check all of the possibilities here https://aiven.io/pricing\n cloudName: google-europe-west1\n plan: startup-4\n\n # general Aiven configuration\n maintenanceWindowDow: friday\n maintenanceWindowTime: 23:00:00\n\n # specific PostgreSQL configuration\n userConfig:\n pg_version: \"11\"\n2. Create the service by applying the configuration:
kubectl apply -f pg-sample.yaml\n3. Review the resource you created with the following command:
kubectl get postgresqls.aiven.io pg-sample\nThe output is similar to the following:
NAME PROJECT REGION PLAN STATE\npg-sample your-project google-europe-west1 startup-4 RUNNING\nThe resource can stay in the BUILDING state for a couple of minutes. Once the state changes to RUNNING, you are ready to access it.
For your convenience, the operator automatically stores the PostgreSQL connection information in a Secret created with the name specified on the connInfoSecretTarget field.
kubectl describe secret pg-connection\nThe output is similar to the following:
Name: pg-connection\nNamespace: default\nAnnotations: <none>\n\nType: Opaque\n\nData\n====\nDATABASE_URI: 107 bytes\nPGDATABASE: 9 bytes\nPGHOST: 38 bytes\nPGPASSWORD: 16 bytes\nPGPORT: 5 bytes\nPGSSLMODE: 7 bytes\nPGUSER: 8 bytes\nYou can use the jq to quickly decode the Secret:
kubectl get secret pg-connection -o json | jq '.data | map_values(@base64d)'\nThe output is similar to the following:
{\n \"DATABASE_URI\": \"postgres://avnadmin:<secret-password>@pg-sample-your-project.aivencloud.com:13039/defaultdb?sslmode=require\",\n \"PGDATABASE\": \"defaultdb\",\n \"PGHOST\": \"pg-sample-your-project.aivencloud.com\",\n \"PGPASSWORD\": \"<secret-password>\",\n \"PGPORT\": \"13039\",\n \"PGSSLMODE\": \"require\",\n \"PGUSER\": \"avnadmin\"\n}\nYou can verify your PostgreSQL connection from a Kubernetes workload by deploying a Pod that runs the psql command.
1. Create a file named pod-psql.yaml
apiVersion: v1\nkind: Pod\nmetadata:\n name: psql-test-connection\nspec:\n restartPolicy: Never\n containers:\n - image: postgres:11-alpine\n name: postgres\n command: [\"psql\", \"$(DATABASE_URI)\", \"-c\", \"SELECT version();\"]\n\n # the pg-connection Secret becomes environment variables\n envFrom:\n - secretRef:\n name: pg-connection\nIt runs once and stops, due to the restartPolicy: Never flag.
2. Inspect the log:
kubectl logs psql-test-connection\nThe output is similar to the following:
version\n---------------------------------------------------------------------------------------------\n PostgreSQL 11.12 on x86_64-pc-linux-gnu, compiled by gcc, a 68c5366192 p 6b9244f01a, 64-bit\n(1 row)\nYou have now connected to the PostgreSQL, and executed the SELECT version(); query.
The Database Kubernetes resource allows you to create a logical database within the PostgreSQL instance.
Create the pg-database-sample.yaml file with the following content:
apiVersion: aiven.io/v1alpha1\nkind: Database\nmetadata:\n name: pg-database-sample\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n # the name of the previously created PostgreSQL instance\n serviceName: pg-sample\n\n project: PROJECT_NAME\n lcCollate: en_US.UTF-8\n lcCtype: en_US.UTF-8\nYou can now connect to the pg-database-sample using the credentials stored in the pg-connection Secret.
Aiven uses the concept of service user that allows you to create users for different services. You can create one for the PostgreSQL instance.
1. Create a file named pg-service-user.yaml.
apiVersion: aiven.io/v1alpha1\nkind: ServiceUser\nmetadata:\n name: pg-service-user\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n connInfoSecretTarget:\n name: pg-service-user-connection\n\n project: PROJECT_NAME\n serviceName: pg-sample\n2. Apply the configuration with the following command.
kubectl apply -f pg-service-user.yaml\nThe ServiceUser resource generates a Secret with connection information, in this case named pg-service-user-connection:
kubectl get secret pg-service-user-connection -o json | jq '.data | map_values(@base64d)'\nThe output has the password and username:
{\n \"PASSWORD\": \"<secret-password>\",\n \"USERNAME\": \"pg-service-user\"\n}\nYou can now connect to the PostgreSQL instance using the credentials generated above, and the host information from the pg-connection Secret.
Connection pooling allows you to maintain very large numbers of connections to a database while minimizing the consumption of server resources. For more information, refer to the connection pooling article in Aiven Docs. Aiven for PostgreSQL uses PGBouncer for connection pooling.
You can create a connection pool with the ConnectionPool resource using the previously created Database and ServiceUser:
Create a new file named pg-connection-pool.yaml with the following content:
apiVersion: aiven.io/v1alpha1\nkind: ConnectionPool\nmetadata:\n name: pg-connection-pool\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n connInfoSecretTarget:\n name: pg-connection-pool-connection\n\n project: PROJECT_NAME\n serviceName: pg-sample\n databaseName: pg-database-sample\n username: pg-service-user\n poolSize: 10\n poolMode: transaction\nThe ConnectionPool generates a Secret with the connection info using the name from the connInfoSecretTarget.Name field:
kubectl get secret pg-connection-pool-connection -o json | jq '.data | map_values(@base64d)'\nThe output is similar to the following:
{\n \"DATABASE_URI\": \"postgres://pg-service-user:<secret-password>@pg-sample-you-project.aivencloud.com:13040/pg-connection-pool?sslmode=require\",\n \"PGDATABASE\": \"pg-database-sample\",\n \"PGHOST\": \"pg-sample-your-project.aivencloud.com\",\n \"PGPASSWORD\": \"<secret-password>\",\n \"PGPORT\": \"13040\",\n \"PGSSLMODE\": \"require\",\n \"PGUSER\": \"pg-service-user\"\n}\nRead-only replicas can be used to reduce the load on the primary service by making read-only queries against the replica service.
To create a read-only replica for a PostgreSQL service, you create a second PostgreSQL service and use serviceIntegrations to replicate data from your primary service.
The example that follows creates a primary service and a read-only replica.
1. Create a new file named pg-read-replica.yaml with the following:
apiVersion: aiven.io/v1alpha1\nkind: PostgreSQL\nmetadata:\n name: primary-pg-service\nspec:\n # gets the authentication token from the `aiven-token` Secret\n authSecretRef:\n name: aiven-token\n key: token\n\n # add your project's name here\n project: PROJECT_NAME\n\n # add the cloud provider and plan of your choice\n # you can see all of the options at https://aiven.io/pricing\n cloudName: google-europe-west1\n plan: startup-4\n\n # general Aiven configuration\n maintenanceWindowDow: friday\n maintenanceWindowTime: 23:00:00\n userConfig:\n pg_version: \"15\"\n\n---\napiVersion: aiven.io/v1alpha1\nkind: PostgreSQL\nmetadata:\n name: read-replica-pg\nspec:\n # gets the authentication token from the `aiven-token` Secret\n authSecretRef:\n name: aiven-token\n key: token\n\n # add your project's name here\n project: PROJECT_NAME\n\n # add the cloud provider and plan of your choice\n # you can see all of the options at https://aiven.io/pricing\n cloudName: google-europe-west1\n plan: startup-4\n\n # general Aiven configuration\n maintenanceWindowDow: saturday\n maintenanceWindowTime: 23:00:00\n userConfig:\n pg_version: \"15\"\n\n # use the read_replica integration and point it to your primary service\n serviceIntegrations:\n - integrationType: read_replica\n sourceServiceName: primary-pg-service\nNote
You can create the replica service in a different region or on a different cloud provider.
2. Apply the configuration with the following command:
kubectl apply -f pg-read-replica.yaml\nThe output is similar to the following:
postgresql.aiven.io/primary-pg-service created\npostgresql.aiven.io/read-replica-pg created\n3. Check the status of the primary service with the following command:
kubectl get postgresqls.aiven.io primary-pg-service\nThe output is similar to the following:
NAME PROJECT REGION PLAN STATE\nprimary-pg-service PROJECT_NAME google-europe-west1 startup-4 RUNNING\nThe resource can be in the BUILDING state for a few minutes. After the state of the primary service changes to RUNNING, the read-only replica is created. You can check the status of the replica using the same command with the name of the replica:
kubectl get postgresqls.aiven.io read-replica-pg\nVirtual Private Cloud (VPC) peering is a method of connecting separate AWS, Google Cloud or Microsoft Azure private networks to each other. It makes it possible for the virtual machines in the different VPCs to talk to each other directly without going through the public internet.
Within the Aiven Kubernetes Operator, you can create a ProjectVPC on Aiven's side to connect to your cloud provider.
Note
Before going through this guide, make sure you have a Kubernetes cluster with the operator installed (see instructions for helm or kubectl), and a Kubernetes Secret with an Aiven authentication token.
"},{"location":"resources/project-vpc.html#create-an-aiven-vpc","title":"Create an Aiven VPC","text":"1. Create a file named vpc-sample.yaml with the following content:
apiVersion: aiven.io/v1alpha1\nkind: ProjectVPC\nmetadata:\n name: vpc-sample\nspec:\n # gets the authentication token from the `aiven-token` Secret\n authSecretRef:\n name: aiven-token\n key: token\n\n project: PROJECT_NAME\n\n # creates a VPC to link an AWS account on the South Africa region\n cloudName: aws-af-south-1\n\n # the network range used by the VPC\n networkCidr: 192.168.0.0/24\n2. Create the Project by applying the configuration:
kubectl apply -f vpc-sample.yaml\n3. Review the resource you created with the following command:
kubectl get projects.aiven.io vpc-sample\nThe output is similar to the following:
NAME PROJECT CLOUD NETWORK CIDR\nvpc-sample PROJECT_NAME aws-af-south-1 192.168.0.0/24\nFollow the official VPC documentation to complete the VPC peering on your cloud of choice.
"},{"location":"resources/project.html","title":"Aiven Project","text":"Note
Before going through this guide, make sure you have a Kubernetes cluster with the operator installed (see instructions for helm or kubectl), and a Kubernetes Secret with an Aiven authentication token.
The Project CRD allows you to create Aiven Projects, where your resources can be located.
To create a fully working Aiven Project with the Aiven Operator you need a source Aiven Project already created with a working billing configuration, like a credit card.
Create a file named project-sample.yaml with the following content:
apiVersion: aiven.io/v1alpha1\nkind: Project\nmetadata:\n name: project-sample\nspec:\n # the source Project to copy the billing information from\n copyFromProject: SOURCE_PROJECT\n\n authSecretRef:\n name: aiven-token\n key: token\n\n connInfoSecretTarget:\n name: project-sample\nApply the resource with:
kubectl apply -f project-sample.yaml\nVerify the newly created Project:
kubectl get projects.aiven.io project-sample\nThe output is similar to the following:
NAME AGE\nproject-sample 22s\nAiven for Redis\u00ae* is a fully managed in-memory NoSQL database that you can deploy in the cloud of your choice to store and access data quickly and efficiently.
Note
Before going through this guide, make sure you have a Kubernetes cluster with the operator installed (see instructions for helm or kubectl), and a Kubernetes Secret with an Aiven authentication token.
"},{"location":"resources/redis.html#create-a-redis-instance","title":"Create a Redis instance","text":"1. Create a file named redis-sample.yaml, and add the following content:
apiVersion: aiven.io/v1alpha1\nkind: Redis\nmetadata:\n name: redis-sample\nspec:\n # gets the authentication token from the `aiven-token` Secret\n authSecretRef:\n name: aiven-token\n key: token\n\n # outputs the Redis connection on the `redis-secret` Secret\n connInfoSecretTarget:\n name: redis-secret\n\n # add your Project name here\n project: PROJECT_NAME\n\n # cloud provider and plan of your choice\n # you can check all of the possibilities here https://aiven.io/pricing\n cloudName: google-europe-west1\n plan: startup-4\n\n # general Aiven configuration\n maintenanceWindowDow: friday\n maintenanceWindowTime: 23:00:00\n\n # specific Redis configuration\n userConfig:\n redis_maxmemory_policy: \"allkeys-random\"\n2. Create the service by applying the configuration:
kubectl apply -f redis-sample.yaml\n3. Review the resource you created with this command:
kubectl describe redis.aiven.io redis-sample\nThe output is similar to the following:
...\nStatus:\n Conditions:\n Last Transition Time: 2023-01-19T14:48:59Z\n Message: Successfully created or updated the instance in Aiven\n Reason: Created\n Status: True\n Type: Initialized\n Last Transition Time: 2023-01-19T14:48:59Z\n Message: Successfully created or updated the instance in Aiven, status remains unknown\n Reason: Created\n Status: Unknown\n Type: Running\n State: REBUILDING\n...\nThe resource will be in the REBUILDING state for a few minutes. Once the state changes to RUNNING, you can access the resource.
For your convenience, the operator automatically stores the Redis connection information in a Secret created with the name specified on the connInfoSecretTarget field.
To view the details of the Secret, use the following command:
kubectl describe secret redis-secret\nThe output is similar to the following:
Name: redis-secret\nNamespace: default\nLabels: <none>\nAnnotations: <none>\n\nType: Opaque\n\nData\n====\nSSL: 8 bytes\nUSER: 7 bytes\nHOST: 60 bytes\nPASSWORD: 24 bytes\nPORT: 5 bytes\nYou can use the jq to quickly decode the Secret:
kubectl get secret redis-secret -o json | jq '.data | map_values(@base64d)'\nThe output is similar to the following:
{\n \"HOST\": \"redis-sample-your-project.aivencloud.com\",\n \"PASSWORD\": \"<secret-password>\",\n \"PORT\": \"14610\",\n \"SSL\": \"required\",\n \"USER\": \"default\"\n}\nService Integrations provide additional functionality and features by connecting different Aiven services together.
See our Getting Started with Service Integrations guide for more information.
Note
Before going through this guide, make sure you have a Kubernetes cluster with the operator installed (see instructions for helm or kubectl), and a Kubernetes Secret with an Aiven authentication token.
"},{"location":"resources/service-integrations.html#send-kafka-logs-to-a-kafka-topic","title":"Send Kafka logs to a Kafka Topic","text":"This integration allows you to send Kafka service logs to a specific Kafka Topic.
First, let's create a Kafka service and a topic.
1. Create a new file named kafka-sample-topic.yaml with the following content:
apiVersion: aiven.io/v1alpha1\nkind: Kafka\nmetadata:\n name: kafka-sample\nspec:\n # gets the authentication token from the `aiven-token` Secret\n authSecretRef:\n name: aiven-token\n key: token\n\n # outputs the Kafka connection on the `kafka-connection` Secret\n connInfoSecretTarget:\n name: kafka-auth\n\n # add your Project name here\n project: PROJECT_NAME\n\n # cloud provider and plan of your choice\n # you can check all of the possibilities here https://aiven.io/pricing\n cloudName: google-europe-west1\n plan: startup-2\n\n # general Aiven configuration\n maintenanceWindowDow: friday\n maintenanceWindowTime: 23:00:00\n\n # specific Kafka configuration\n userConfig:\n kafka_version: \"2.7\"\n\n---\napiVersion: aiven.io/v1alpha1\nkind: KafkaTopic\nmetadata:\n name: logs\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n project: PROJECT_NAME\n serviceName: kafka-sample\n\n # here we can specify how many partitions the topic should have\n partitions: 3\n # and the topic replication factor\n replication: 2\n\n # we also support various topic-specific configurations\n config:\n flush_ms: 100\n2. Create the resource on Kubernetes:
kubectl apply -f kafka-sample-topic.yaml\n3. Now, create a ServiceIntegration resource to send the Kafka logs to the created topic. In the same file, add the following YAML:
apiVersion: aiven.io/v1alpha1\nkind: ServiceIntegration\nmetadata:\n name: service-integration-kafka-logs\nspec:\n # gets the authentication token from the `aiven-token` Secret\n authSecretRef:\n name: aiven-token\n key: token\n\n project: PROJECT_NAME\n\n # indicates the type of the integration\n integrationType: kafka_logs\n\n # we will send the logs to the same kafka-sample instance\n # the source and destination are the same\n sourceServiceName: kafka-sample\n destinationServiceName: kafka-sample\n\n # the topic name we will send to\n kafkaLogs:\n kafka_topic: logs\n4. Reapply the resource on Kubernetes:
kubectl apply -f kafka-sample-topic.yaml\n5. Let's check the created service integration:
kubectl get serviceintegrations.aiven.io service-integration-kafka-logs\nThe output is similar to the following:
NAME PROJECT TYPE SOURCE SERVICE NAME DESTINATION SERVICE NAME SOURCE ENDPOINT ID DESTINATION ENDPOINT ID\nservice-integration-kafka-logs your-project kafka_logs kafka-sample kafka-sample\nYour Kafka service logs are now being streamed to the logs Kafka topic.
Aiven for Apache Kafka is an excellent option if you need to run Apache Kafka at scale. With Aiven Kubernetes Operator you can get up and running with a suitably sized Apache Kafka service in a few minutes.
Note
Before going through this guide, make sure you have a Kubernetes cluster with the operator installed (see instructions for helm or kubectl), and a Kubernetes Secret with an Aiven authentication token.
"},{"location":"resources/kafka/index.html#create-a-kafka-instance","title":"Create a Kafka instance","text":"1. Create a file named kafka-sample.yaml, and add the following content:
apiVersion: aiven.io/v1alpha1\nkind: Kafka\nmetadata:\n name: kafka-sample\nspec:\n # gets the authentication token from the `aiven-token` Secret\n authSecretRef:\n name: aiven-token\n key: token\n\n # outputs the Kafka connection on the `kafka-connection` Secret\n connInfoSecretTarget:\n name: kafka-auth\n\n # add your Project name here\n project: PROJECT_NAME\n\n # cloud provider and plan of your choice\n # you can check all of the possibilities here https://aiven.io/pricing\n cloudName: google-europe-west1\n plan: startup-2\n\n # general Aiven configuration\n maintenanceWindowDow: friday\n maintenanceWindowTime: 23:00:00\n\n # specific Kafka configuration\n userConfig:\n kafka_version: \"2.7\"\n2. Create the following resource on Kubernetes:
kubectl apply -f kafka-sample.yaml\n3. Inspect the service created using the command below.
kubectl get kafka.aiven.io kafka-sample\nThe output has the project name and state, similar to the following:
NAME PROJECT REGION PLAN STATE\nkafka-sample PROJECT_NAME google-europe-west1 startup-2 RUNNING\nAfter a couple of minutes, the STATE field is changed to RUNNING, and is ready to be used.
For your convenience, the operator automatically stores the Kafka connection information in a Secret created with the name specified on the connInfoSecretTarget field.
kubectl describe secret kafka-auth\nThe output is similar to the following:
Name: kafka-auth\nNamespace: default\nAnnotations: <none>\n\nType: Opaque\n\nData\n====\nCA_CERT: 1537 bytes\nHOST: 41 bytes\nPASSWORD: 16 bytes\nPORT: 5 bytes\nUSERNAME: 8 bytes\nACCESS_CERT: 1533 bytes\nACCESS_KEY: 2484 bytes\nYou can use the jq to quickly decode the Secret:
kubectl get secret kafka-auth -o json | jq '.data | map_values(@base64d)'\nThe output is similar to the following:
{\n \"CA_CERT\": \"<secret-ca-cert>\",\n \"ACCESS_CERT\": \"<secret-cert>\",\n \"ACCESS_KEY\": \"<secret-access-key>\",\n \"HOST\": \"kafka-sample-your-project.aivencloud.com\",\n \"PASSWORD\": \"<secret-password>\",\n \"PORT\": \"13041\",\n \"USERNAME\": \"avnadmin\"\n}\nYou can verify your access to the Kafka cluster from a Pod using the authentication data from the kafka-auth Secret. kcat is used for our examples below.
1. Create a file named kafka-test-connection.yaml, and add the following content:
apiVersion: v1\nkind: Pod\nmetadata:\n name: kafka-test-connection\nspec:\n restartPolicy: Never\n containers:\n - image: edenhill/kcat:1.7.0\n name: kcat\n\n # the command below will connect to the Kafka cluster\n # and output its metadata\n command:\n [\n \"kcat\",\n \"-b\",\n \"$(HOST):$(PORT)\",\n \"-X\",\n \"security.protocol=SSL\",\n \"-X\",\n \"ssl.key.location=/kafka-auth/ACCESS_KEY\",\n \"-X\",\n \"ssl.key.password=$(PASSWORD)\",\n \"-X\",\n \"ssl.certificate.location=/kafka-auth/ACCESS_CERT\",\n \"-X\",\n \"ssl.ca.location=/kafka-auth/CA_CERT\",\n \"-L\",\n ]\n\n # loading the data from the Secret as environment variables\n # useful to access the Kafka information, like hostname and port\n envFrom:\n - secretRef:\n name: kafka-auth\n\n volumeMounts:\n - name: kafka-auth\n mountPath: \"/kafka-auth\"\n\n # loading the data from the Secret as files in a volume\n # useful to access the Kafka certificates\n volumes:\n - name: kafka-auth\n secret:\n secretName: kafka-auth\n2. Apply the file.
kubectl apply -f kafka-test-connection.yaml\nOnce successfully applied, you have a log with the metadata information about the Kafka cluster.
kubectl logs kafka-test-connection\nThe output is similar to the following:
Metadata for all topics (from broker -1: ssl://kafka-sample-your-project.aivencloud.com:13041/bootstrap):\n 3 brokers:\n broker 2 at 35.205.234.70:13041\n broker 3 at 34.77.127.70:13041 (controller)\n broker 1 at 34.78.146.156:13041\n 0 topics:\nKafkaTopic and KafkaACL","text":"To properly produce and consume content on Kafka, you need topics and ACLs. The operator supports both with the KafkaTopic and KafkaACL resources.
Below, here is how to create a Kafka topic named random-strings where random string messages will be sent.
1. Create a file named kafka-topic-random-strings.yaml with the content below:
apiVersion: aiven.io/v1alpha1\nkind: KafkaTopic\nmetadata:\n name: random-strings\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n project: PROJECT_NAME\n serviceName: kafka-sample\n\n # here we can specify how many partitions the topic should have\n partitions: 3\n # and the topic replication factor\n replication: 2\n\n # we also support various topic-specific configurations\n config:\n flush_ms: 100\n2. Create the resource on Kubernetes:
kubectl apply -f kafka-topic-random-strings.yaml\n3. Create a user and an ACL. To use the Kafka topic, create a new user with the ServiceUser resource (in order to avoid using the avnadmin superuser), and the KafkaACL to allow the user access to the topic.
In a file named kafka-acl-user-crab.yaml, add the following two resources:
apiVersion: aiven.io/v1alpha1\nkind: ServiceUser\nmetadata:\n # the name of our user \ud83e\udd80\n name: crab\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n # the Secret name we will store the users' connection information\n # looks exactly the same as the Secret generated when creating the Kafka cluster\n # we will use this Secret to produce and consume events later!\n connInfoSecretTarget:\n name: kafka-crab-connection\n\n # the Aiven project the user is related to\n project: PROJECT_NAME\n\n # the name of our Kafka Service\n serviceName: kafka-sample\n\n---\napiVersion: aiven.io/v1alpha1\nkind: KafkaACL\nmetadata:\n name: crab\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n project: PROJECT_NAME\n serviceName: kafka-sample\n\n # the username from the ServiceUser above\n username: crab\n\n # the ACL allows to produce and consume on the topic\n permission: readwrite\n\n # specify the topic we created before\n topic: random-strings\nTo create the crab user and its permissions, execute the following command:
kubectl apply -f kafka-acl-user-crab.yaml\nUsing the previously created KafkaTopic, ServiceUser, KafkaACL, you can produce and consume events.
You can use kcat to produce a message into Kafka, and the -t random-strings argument to select the desired topic, and use the content of the /etc/issue file as the message's body.
1. Create a kafka-crab-produce.yaml file with the content below:
apiVersion: v1\nkind: Pod\nmetadata:\n name: kafka-crab-produce\nspec:\n restartPolicy: Never\n containers:\n - image: edenhill/kcat:1.7.0\n name: kcat\n\n # the command below will produce a message with the /etc/issue file content\n command:\n [\n \"kcat\",\n \"-b\",\n \"$(HOST):$(PORT)\",\n \"-X\",\n \"security.protocol=SSL\",\n \"-X\",\n \"ssl.key.location=/crab-auth/ACCESS_KEY\",\n \"-X\",\n \"ssl.key.password=$(PASSWORD)\",\n \"-X\",\n \"ssl.certificate.location=/crab-auth/ACCESS_CERT\",\n \"-X\",\n \"ssl.ca.location=/crab-auth/CA_CERT\",\n \"-P\",\n \"-t\",\n \"random-strings\",\n \"/etc/issue\",\n ]\n\n # loading the crab user data from the Secret as environment variables\n # useful to access the Kafka information, like hostname and port\n envFrom:\n - secretRef:\n name: kafka-crab-connection\n\n volumeMounts:\n - name: crab-auth\n mountPath: \"/crab-auth\"\n\n # loading the crab user information from the Secret as files in a volume\n # useful to access the Kafka certificates\n volumes:\n - name: crab-auth\n secret:\n secretName: kafka-crab-connection\n2. Create the Pod with the following content:
kubectl apply -f kafka-crab-produce.yaml\nNow your event is stored in Kafka.
To consume a message, you can use a graphical interface called Kowl. It allows you to explore information about our Kafka cluster, such as brokers, topics, or consumer groups.
1. Create a Kubernetes Pod and service to deploy and access Kowl. Create a file named kafka-crab-consume.yaml with the content below:
apiVersion: v1\nkind: Pod\nmetadata:\n name: kafka-crab-consume\n labels:\n app: kafka-crab-consume\nspec:\n containers:\n - image: quay.io/cloudhut/kowl:v1.4.0\n name: kowl\n\n # kowl configuration values\n env:\n - name: KAFKA_TLS_ENABLED\n value: \"true\"\n\n - name: KAFKA_BROKERS\n value: $(HOST):$(PORT)\n - name: KAFKA_TLS_PASSPHRASE\n value: $(PASSWORD)\n\n - name: KAFKA_TLS_CAFILEPATH\n value: /crab-auth/CA_CERT\n - name: KAFKA_TLS_CERTFILEPATH\n value: /crab-auth/ACCESS_CERT\n - name: KAFKA_TLS_KEYFILEPATH\n value: /crab-auth/ACCESS_KEY\n\n # inject all connection information as environment variables\n envFrom:\n - secretRef:\n name: kafka-crab-connection\n\n volumeMounts:\n - name: crab-auth\n mountPath: /crab-auth\n\n # loading the crab user information from the Secret as files in a volume\n # useful to access the Kafka certificates\n volumes:\n - name: crab-auth\n secret:\n secretName: kafka-crab-connection\n\n---\n# we will be using a simple service to access Kowl on port 8080\napiVersion: v1\nkind: Service\nmetadata:\n name: kafka-crab-consume\nspec:\n selector:\n app: kafka-crab-consume\n ports:\n - port: 8080\n targetPort: 8080\n2. Create the resources with:
kubectl apply -f kafka-crab-consume.yaml\n3. In another terminal create a port-forward tunnel to your Pod:
kubectl port-forward kafka-crab-consume 8080:8080\n4. In the browser of your choice, access the http://localhost:8080 address. You now see a page with the random-strings topic listed:
5. Click the topic name to see the message.
You have now consumed the message.
"},{"location":"resources/kafka/connect.html","title":"Kafka Connect","text":"Aiven for Apache Kafka Connect is a framework and a runtime for integrating Kafka with other systems. Kafka connectors can either be a source (for pulling data from other systems into Kafka) or sink (for pushing data into other systems from Kafka).
This section involves a few different Kubernetes CRDs:
KafkaService service with a KafkaTopicKafkaConnect serviceServiceIntegration to integrate the Kafka and KafkaConnect servicesPostgreSQL used as a sink to receive messages from KafkaKafkaConnector to finally connect the Kafka with the PostgreSQLCreate a file named kafka-sample-connect.yaml with the following content:
apiVersion: aiven.io/v1alpha1\nkind: Kafka\nmetadata:\n name: kafka-sample-connect\nspec:\n # gets the authentication token from the `aiven-token` Secret\n authSecretRef:\n name: aiven-token\n key: token\n\n # outputs the Kafka connection on the `kafka-connection` Secret\n connInfoSecretTarget:\n name: kafka-auth\n\n # add your Project name here\n project: PROJECT_NAME\n\n # cloud provider and plan of your choice\n # you can check all of the possibilities here https://aiven.io/pricing\n cloudName: google-europe-west1\n plan: business-4\n\n # general Aiven configuration\n maintenanceWindowDow: friday\n maintenanceWindowTime: 23:00:00\n\n # specific Kafka configuration\n userConfig:\n kafka_version: \"2.7\"\n kafka_connect: true\n\n---\napiVersion: aiven.io/v1alpha1\nkind: KafkaTopic\nmetadata:\n name: kafka-topic-connect\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n project: PROJECT_NAME\n serviceName: kafka-sample-connect\n\n replication: 2\n partitions: 1\nNext, create a file named kafka-connect.yaml and add the following KafkaConnect resource:
apiVersion: aiven.io/v1alpha1\nkind: KafkaConnect\nmetadata:\n name: kafka-connect\nspec:\n # gets the authentication token from the `aiven-token` Secret\n authSecretRef:\n name: aiven-token\n key: token\n\n # add your Project name here\n project: PROJECT_NAME\n\n # cloud provider and plan of your choice\n # you can check all of the possibilities here https://aiven.io/pricing\n cloudName: google-europe-west1\n plan: startup-4\n\n # general Aiven configuration\n maintenanceWindowDow: friday\n maintenanceWindowTime: 23:00:00\nNow let's create a ServiceIntegration. It will use the fields sourceServiceName and destinationServiceName to integrate the previously created kafka-sample-connect and kafka-connect. Open a new file named service-integration-connect.yaml and add the content below:
apiVersion: aiven.io/v1alpha1\nkind: ServiceIntegration\nmetadata:\n name: service-integration-kafka-connect\nspec:\n # gets the authentication token from the `aiven-token` Secret\n authSecretRef:\n name: aiven-token\n key: token\n\n project: PROJECT_NAME\n\n # indicates the type of the integration\n integrationType: kafka_connect\n\n # we will send messages from the `kafka-sample-connect` to `kafka-connect`\n sourceServiceName: kafka-sample-connect\n destinationServiceName: kafka-connect\nLet's add an Aiven for PostgreSQL service. It will be the service used as a sink, receiving messages from the kafka-sample-connect cluster. Create a file named pg-sample-connect.yaml with the content below:
apiVersion: aiven.io/v1alpha1\nkind: PostgreSQL\nmetadata:\n name: pg-connect\nspec:\n # gets the authentication token from the `aiven-token` Secret\n authSecretRef:\n name: aiven-token\n key: token\n\n # outputs the PostgreSQL connection on the `pg-connection` Secret\n connInfoSecretTarget:\n name: pg-connection\n\n # add your Project name here\n project: PROJECT_NAME\n\n # cloud provider and plan of your choice\n # you can check all of the possibilities here https://aiven.io/pricing\n cloudName: google-europe-west1\n plan: startup-4\n\n # general Aiven configuration\n maintenanceWindowDow: friday\n maintenanceWindowTime: 23:00:00\nFinally, let's add the glue of everything: a KafkaConnector. As described in the specification, it will send receive messages from the kafka-sample-connect and send them to the pg-connect service. Check our official documentation for more connectors.
Create a file named kafka-connector-connect.yaml and with the content below:
apiVersion: aiven.io/v1alpha1\nkind: KafkaConnector\nmetadata:\n name: kafka-connector\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n project: PROJECT_NAME\n\n # the Kafka cluster name\n serviceName: kafka-sample-connect\n\n # the connector we will be using\n connectorClass: io.aiven.connect.jdbc.JdbcSinkConnector\n\n userConfig:\n auto.create: \"true\"\n\n # constructs the pg-connect connection information\n connection.url: 'jdbc:postgresql://{{ fromSecret \"pg-connection\" \"PGHOST\"}}:{{ fromSecret \"pg-connection\" \"PGPORT\" }}/{{ fromSecret \"pg-connection\" \"PGDATABASE\" }}'\n connection.user: '{{ fromSecret \"pg-connection\" \"PGUSER\" }}'\n connection.password: '{{ fromSecret \"pg-connection\" \"PGPASSWORD\" }}'\n\n # specify which topics it will watch\n topics: kafka-topic-connect\n\n key.converter: org.apache.kafka.connect.json.JsonConverter\n value.converter: org.apache.kafka.connect.json.JsonConverter\n value.converter.schemas.enable: \"true\"\nWith all the files created, apply the new Kubernetes resources:
kubectl apply \\\n -f kafka-sample-connect.yaml \\\n -f kafka-connect.yaml \\\n -f service-integration-connect.yaml \\\n -f pg-sample-connect.yaml \\\n -f kafka-connector-connect.yaml\nIt will take some time for all the services to be up and running. You can check their status with the following command:
kubectl get \\\n kafkas.aiven.io/kafka-sample-connect \\\n kafkaconnects.aiven.io/kafka-connect \\\n postgresqls.aiven.io/pg-connect \\\n kafkaconnectors.aiven.io/kafka-connector\nThe output is similar to the following:
NAME PROJECT REGION PLAN STATE\nkafka.aiven.io/kafka-sample-connect your-project google-europe-west1 business-4 RUNNING\n\nNAME STATE\nkafkaconnect.aiven.io/kafka-connect RUNNING\n\nNAME PROJECT REGION PLAN STATE\npostgresql.aiven.io/pg-connect your-project google-europe-west1 startup-4 RUNNING\n\nNAME SERVICE NAME PROJECT CONNECTOR CLASS STATE TASKS TOTAL TASKS RUNNING\nkafkaconnector.aiven.io/kafka-connector kafka-sample-connect your-project io.aiven.connect.jdbc.JdbcSinkConnector RUNNING 1 1\nThe deployment is finished when all services have the state RUNNING.
To test the connection integration, let's produce a Kafka message using kcat from within the Kubernetes cluster. We will deploy a Pod responsible for crafting a message and sending to the Kafka cluster, using the kafka-auth secret generate by the Kafka CRD.
Create a new file named kcat-connect.yaml and add the content below:
apiVersion: v1\nkind: Pod\nmetadata:\n name: kafka-message\nspec:\n containers:\n\n restartPolicy: Never\n - image: edenhill/kcat:1.7.0\n name: kcat\n\n command: ['/bin/sh']\n args: [\n '-c',\n 'echo {\\\"schema\\\":{\\\"type\\\":\\\"struct\\\",\\\"fields\\\":[{ \\\"field\\\": \\\"text\\\", \\\"type\\\": \\\"string\\\", \\\"optional\\\": false } ] }, \\\"payload\\\": { \\\"text\\\": \\\"Hello World\\\" } } > /tmp/msg;\n\n kcat\n -b $(HOST):$(PORT)\n -X security.protocol=SSL\n -X ssl.key.location=/kafka-auth/ACCESS_KEY\n -X ssl.key.password=$(PASSWORD)\n -X ssl.certificate.location=/kafka-auth/ACCESS_CERT\n -X ssl.ca.location=/kafka-auth/CA_CERT\n -P -t kafka-topic-connect /tmp/msg'\n ]\n\n envFrom:\n - secretRef:\n name: kafka-auth\n\n volumeMounts:\n - name: kafka-auth\n mountPath: \"/kafka-auth\"\n\n volumes:\n - name: kafka-auth\n secret:\n secretName: kafka-auth\nApply the file with:
kubectl apply -f kcat-connect.yaml\nThe Pod will execute the commands and finish. You can confirm its Completed state with:
kubectl get pod kafka-message\nThe output is similar to the following:
NAME READY STATUS RESTARTS AGE\nkafka-message 0/1 Completed 0 5m35s\nIf everything went smoothly, we should have our produced message in the PostgreSQL service. Let's check that out.
Create a file named psql-connect.yaml with the content below:
apiVersion: v1\nkind: Pod\nmetadata:\n name: psql-connect\nspec:\n restartPolicy: Never\n containers:\n - image: postgres:13\n name: postgres\n # \"kafka-topic-connect\" is the table automatically created by KafkaConnect\n command:\n [\n \"psql\",\n \"$(DATABASE_URI)\",\n \"-c\",\n 'SELECT * from \"kafka-topic-connect\";',\n ]\n\n envFrom:\n - secretRef:\n name: pg-connection\nApply the file with:
kubectl apply -f psql-connect.yaml\nAfter a couple of seconds, inspect its log with this command:
kubectl logs psql-connect\nThe output is similar to the following:
text\n-------------\n Hello World\n(1 row)\nTo clean up all the created resources, use the following command:
kubectl delete \\\n -f kafka-sample-connect.yaml \\\n -f kafka-connect.yaml \\\n -f service-integration-connect.yaml \\\n -f pg-sample-connect.yaml \\\n -f kafka-connector-connect.yaml \\\n -f kcat-connect.yaml \\\n -f psql-connect.yaml\nKafkaSchema","text":"Aiven develops and maintain Karapace, an open source implementation of Kafka REST and schema registry. Is available out of the box for our managed Kafka service.
The schema registry address and authentication is the same as the Kafka broker, the only different is the usage of the port 13044.
First, let's create an Aiven for Apache Kafka service.
1. Create a file named kafka-sample-schema.yaml and add the content below:
apiVersion: aiven.io/v1alpha1\nkind: Kafka\nmetadata:\n name: kafka-sample-schema\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n connInfoSecretTarget:\n name: kafka-auth\n\n project: PROJECT_NAME\n cloudName: google-europe-west1\n plan: startup-2\n maintenanceWindowDow: friday\n maintenanceWindowTime: 23:00:00\n\n userConfig:\n kafka_version: \"2.7\"\n\n # this flag enables the Schema registry\n schema_registry: true\n2. Apply the changes with the following command:
kubectl apply -f kafka-schema.yaml\nNow, let's create the schema itself.
1. Create a new file named kafka-sample-schema.yaml and add the YAML content below:
apiVersion: aiven.io/v1alpha1\nkind: KafkaSchema\nmetadata:\n name: kafka-schema\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n project: PROJECT_NAME\n serviceName: kafka-sample-schema\n\n # the name of the Schema\n subjectName: MySchema\n\n # the schema itself, in JSON format\n schema: |\n {\n \"type\": \"record\",\n \"name\": \"MySchema\",\n \"fields\": [\n {\n \"name\": \"field\",\n \"type\": \"string\"\n }\n ]\n }\n\n # sets the schema compatibility level\n compatibilityLevel: BACKWARD\n2. Create the schema with the command:
kubectl apply -f kafka-schema.yaml\n3. Review the resource you created with the following command:
kubectl get kafkaschemas.aiven.io kafka-schema\nThe output is similar to the following:
NAME SERVICE NAME PROJECT SUBJECT COMPATIBILITY LEVEL VERSION\nkafka-schema kafka-sample PROJECT_NAME MySchema BACKWARD 1\nNow you can follow the instructions to use a schema registry in Java on how to use the schema created.
"}]} \ No newline at end of file +{"config":{"lang":["en"],"separator":"[\\s\\-]+","pipeline":["stopWordFilter"]},"docs":[{"location":"index.html","title":"Welcome to Aiven Operator for Kubernetes","text":"Provision and manage Aiven services from your Kubernetes cluster.
"},{"location":"index.html#what-is-aiven","title":"What is Aiven?","text":"Aiven offers managed services for the best open source data technologies, on a cloud of your choice.
We offer multiple cloud options because we believe that everyone should have access to great data platforms wherever they host their applications. Our customers tell us they love it because they know that they aren\u2019t locked in to one particular cloud platform for all their data needs.
"},{"location":"index.html#contributing","title":"Contributing","text":"The contribution guide covers everything you need to know about how you can contribute to Aiven Operator for Kubernetes. The developer guide will help you onboard as a developer.
"},{"location":"authentication.html","title":"Authenticating","text":"To get authenticated and authorized, set up the communication between the Aiven Operator for Kubernetes and Aiven by using a token stored in a Kubernetes Secret. You can then refer to the Secret name on every custom resource in the authSecretRef field.
Important
If you don't have an Aiven account yet, sign up here for a free trial \ud83e\udd80
1. Generate an authentication token
Refer to our documentation article to generate your authentication token.
2. Create the Kubernetes Secret
The following command creates a Secret named aiven-token with a token field containing the authentication token:
kubectl create secret generic aiven-token --from-literal=token=\"TOKEN\"\nWhen managing your Aiven resources, we will be using the created Secret in the authSecretRef field. It will look like the example below:
apiVersion: aiven.io/v1alpha1\nkind: PostgreSQL\nmetadata:\n name: pg-sample\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n [ ... ]\nAlso, note that within Aiven, all resources are conceptually inside a Project. By default, a random project name is generated when you signup, but you can also create new projects.
The Project name is required in most of the resources. It will look like the example below:
apiVersion: aiven.io/v1alpha1\nkind: PostgreSQL\nmetadata:\n name: pg-sample\nspec:\n project: PROJECT_NAME\n [ ... ]\nPostgreSQL: wait for a valid backup to create read replicaClickhouseGrant: grant privileges for an unknown table (Clickhouse can do that)ClickhouseGrant: track the state to revoke only known privilegesCassandra field userConfig.cassandra.read_request_timeout_in_ms, type integer: How long the coordinator waits for read operations to complete before timing it outCassandra field userConfig.cassandra.write_request_timeout_in_ms, type integer: How long the coordinator waits for write requests to complete with at least one node in the local datacenterOpenSearch field userConfig.opensearch.knn_memory_circuit_breaker_enabled, type boolean: Enable or disable KNN memory circuit breaker. Defaults to trueOpenSearch field userConfig.opensearch.knn_memory_circuit_breaker_limit, type integer: Maximum amount of memory that can be used for KNN index. Defaults to 50% of the JVM heap sizePostgreSQL field userConfig.pg.log_line_prefix: enum ['%m [%p] %q[user=%u,db=%d,app=%a] ', '%t [%p]: [%l-1] user=%u,db=%d,app=%a,client=%h ', 'pid=%p,user=%u,db=%d,app=%a,client=%h '] \u2192 ['%m [%p] %q[user=%u,db=%d,app=%a] ', '%t [%p]: [%l-1] user=%u,db=%d,app=%a,client=%h ', 'pid=%p,user=%u,db=%d,app=%a,client=%h ', 'pid=%p,user=%u,db=%d,app=%a,client=%h,txid=%x,qid=%Q ']http.StatusBadRequest on ClickhouseGrant deletionClickhouseGrant invalid remote and local privileges comparisonClickhouseGrant: doesn't escape role name to grantClickhouseUser: password was reset due to an incorrect processing cycleClickhouseRole deletion error (missing database)ClickhouseGrant deletion errors (missing database, service, role)REBALANCING stateClickhouseGrantKafkaConnect field userConfig.secret_providers, type array: Configure external secret providers in order to reference external secrets in connector configurationKafka field userConfig.kafka_connect_secret_providers, type array: Configure external secret providers in order to reference external secrets in connector configurationKafka field userConfig.letsencrypt_sasl_privatelink, type boolean: Use Letsencrypt CA for Kafka SASL via PrivatelinkServiceIntegration field datadog.mirrormaker_custom_metrics, type array: List of custom metricsServiceIntegration field kafkaMirrormaker.kafka_mirrormaker.consumer_auto_offset_reset, type string: Set where consumer starts to consume dataServiceIntegration field kafkaMirrormaker.kafka_mirrormaker.consumer_max_poll_records, type integer: Set consumer max.poll.records. The default is 500PostgreSQL field userConfig.pgaudit: deprecatedServiceIntegrationEndpoint field externalPostgresql.ssl_mode: enum [allow, disable, prefer, require, verify-ca, verify-full] \u2192 [require, verify-ca, verify-full]ServiceIntegrationEndpointServiceIntegration flink_external_postgresql typeServiceIntegration field datadog.datadog_pgbouncer_enabled, type boolean: Enable Datadog PgBouncer Metric TrackingServiceIntegration deletion when instance has no id setdisk_space field validationproject, serviceName fields validationConnectionPool doesn't check service user preconditionCA_CERT secret key for Grafana, OpenSearch, Redis, and Clickhouse. Can't be used with these service types ddog-gov.com, us3.datadoghq.com, us5.datadoghq.com]`ServiceIntegrationEndpoint field externalKafka.ssl_endpoint_identification_algorithm: enum [, https] \u2192 [https]ClickhouseUser webhook. Doesn't do any validation or mutationKafka field userConfig.kafka_version: enum [3.4, 3.5, 3.6] \u2192 [3.4, 3.5, 3.6, 3.7]ServiceIntegrationEndpoint field datadog.site: enum [datadoghq.com, datadoghq.eu, ddog-gov.com, us3.datadoghq.com, us5.datadoghq.com] \u2192 `[ap1.datadoghq.com, datadoghq.com, datadoghq.eu,ClickhouseRoleKafkaACLKafkaSchemaRegistryACL kindClickhouseDatabase kindDatabase kind validations and default values with CRD validation rulesREDIS_CA_CERT, MYSQL_CA_CERT, etc.KafkaTopic field config.local_retention_bytes, type integer: local.retention.bytes valueKafkaTopic field config.local_retention_ms, type integer: local.retention.ms valueKafkaTopic field config.remote_storage_enable, type boolean: remote_storage_enableCassandra field userConfig.cassandra_version: pattern ^[0-9]+(\\.[0-9]+)?$Cassandra field userConfig.project_to_fork_from: pattern ^[a-z][-a-z0-9]{0,63}$|^$Cassandra field userConfig.service_to_fork_from: pattern ^[a-z][-a-z0-9]{0,63}$|^$Cassandra field userConfig.service_to_join_with: pattern ^[a-z][-a-z0-9]{0,63}$Clickhouse field userConfig.project_to_fork_from: pattern ^[a-z][-a-z0-9]{0,63}$|^$Clickhouse field userConfig.service_to_fork_from: pattern ^[a-z][-a-z0-9]{0,63}$|^$Grafana field userConfig.project_to_fork_from: pattern ^[a-z][-a-z0-9]{0,63}$|^$Grafana field userConfig.service_to_fork_from: pattern ^[a-z][-a-z0-9]{0,63}$|^$Kafka field userConfig.kafka.sasl_oauthbearer_expected_audience: pattern ^[^\\r\\n]*$Kafka field userConfig.kafka.sasl_oauthbearer_expected_issuer: pattern ^[^\\r\\n]*$Kafka field userConfig.kafka.sasl_oauthbearer_sub_claim_name: pattern ^[^\\r\\n]*$MySQL field userConfig.mysql.default_time_zone: pattern ^([-+][\\d:]*|[\\w/]*)$MySQL field userConfig.project_to_fork_from: pattern ^[a-z][-a-z0-9]{0,63}$|^$MySQL field userConfig.service_to_fork_from: pattern ^[a-z][-a-z0-9]{0,63}$|^$OpenSearch field userConfig.openid.client_id: pattern ^[^\\r\\n]*$OpenSearch field userConfig.openid.client_secret: pattern ^[^\\r\\n]*$OpenSearch field userConfig.openid.header: pattern ^[^\\r\\n]*$OpenSearch field userConfig.openid.jwt_header: pattern ^[^\\r\\n]*$OpenSearch field userConfig.openid.jwt_url_parameter: pattern ^[^\\r\\n]*$OpenSearch field userConfig.openid.roles_key: pattern ^[^\\r\\n]*$OpenSearch field userConfig.openid.scope: pattern ^[^\\r\\n]*$OpenSearch field userConfig.openid.subject_key: pattern ^[^\\r\\n]*$OpenSearch field userConfig.project_to_fork_from: pattern ^[a-z][-a-z0-9]{0,63}$|^$OpenSearch field userConfig.saml.idp_entity_id: pattern ^[^\\r\\n]*$OpenSearch field userConfig.saml.roles_key: pattern ^[^\\r\\n]*$OpenSearch field userConfig.saml.sp_entity_id: pattern ^[^\\r\\n]*$OpenSearch field userConfig.saml.subject_key: pattern ^[^\\r\\n]*$OpenSearch field userConfig.service_to_fork_from: pattern ^[a-z][-a-z0-9]{0,63}$|^$PostgreSQL field userConfig.pg.timezone: pattern ^[\\w/]*$PostgreSQL field userConfig.pg_service_to_fork_from: pattern ^[a-z][-a-z0-9]{0,63}$|^$PostgreSQL field userConfig.project_to_fork_from: pattern ^[a-z][-a-z0-9]{0,63}$|^$PostgreSQL field userConfig.service_to_fork_from: pattern ^[a-z][-a-z0-9]{0,63}$|^$Redis field userConfig.project_to_fork_from: pattern ^[a-z][-a-z0-9]{0,63}$|^$Redis field userConfig.service_to_fork_from: pattern ^[a-z][-a-z0-9]{0,63}$|^$OpenSearch field userConfig.opensearch.plugins_alerting_filter_by_backend_roles, type boolean: Enable or disable filtering of alerting by backend roles. Requires Security pluginRedis field userConfig.redis_notify_keyspace_events: pattern ^[KEg\\$lshzxeA]*$ \u2192 ^[KEg\\$lshzxentdmA]*$PostgreSQL field userConfig.pgaudit, type object: System-wide settings for the pgaudit extensionServiceIntegration field datadog.opensearch.cluster_stats_enabled, type boolean: Enable Datadog Opensearch Cluster MonitoringOpenSearch field userConfig.opensearch.enable_security_audit, type boolean: Enable/Disable security auditKafka field userConfig.kafka_rest_config.name_strategy, type string: Name strategy to use when selecting subject for storing schemasRedis field userConfig.redis_version, type string: Redis major versionGrafana field userConfig.auth_github.auto_login, type boolean: Allow users to bypass the login screen and automatically log inGrafana field userConfig.auth_github.skip_org_role_sync, type boolean: Stop automatically syncing user rolesClickhouse field userConfig.additional_backup_regions: deprecatedGrafana field userConfig.additional_backup_regions: deprecatedKafkaConnect field userConfig.additional_backup_regions: deprecatedKafka field userConfig.additional_backup_regions: deprecatedOpenSearch field userConfig.additional_backup_regions: deprecatedRedis field userConfig.additional_backup_regions: deprecatedCassandra field userConfig.cassandra_version: enum [3, 4, 4.1] \u2192 [4, 4.1]Kafka field userConfig.kafka_version: enum [3.1, 3.3, 3.4, 3.5, 3.6] \u2192 [3.4, 3.5, 3.6]PostgreSQL field userConfig.pg_version: enum [11, 12, 13, 14, 15, 16] \u2192 [12, 13, 14, 15, 16]Cassandra field technicalEmails, type array: Defines the email addresses that will receive alerts about upcoming maintenance updates or warnings about service instabilityClickhouse field technicalEmails, type array: Defines the email addresses that will receive alerts about upcoming maintenance updates or warnings about service instabilityGrafana field technicalEmails, type array: Defines the email addresses that will receive alerts about upcoming maintenance updates or warnings about service instabilityKafkaConnect field technicalEmails, type array: Defines the email addresses that will receive alerts about upcoming maintenance updates or warnings about service instabilityKafka field technicalEmails, type array: Defines the email addresses that will receive alerts about upcoming maintenance updates or warnings about service instabilityMySQL field technicalEmails, type array: Defines the email addresses that will receive alerts about upcoming maintenance updates or warnings about service instabilityOpenSearch field technicalEmails, type array: Defines the email addresses that will receive alerts about upcoming maintenance updates or warnings about service instabilityPostgreSQL field technicalEmails, type array: Defines the email addresses that will receive alerts about upcoming maintenance updates or warnings about service instabilityRedis field technicalEmails, type array: Defines the email addresses that will receive alerts about upcoming maintenance updates or warnings about service instabilityCassandra field connInfoSecretTargetDisabled, type boolean: When true, the secret containing connection information will not be created, defaults to falseClickhouse field connInfoSecretTargetDisabled, type boolean: When true, the secret containing connection information will not be created, defaults to falseClickhouseUser field connInfoSecretTargetDisabled, type boolean: When true, the secret containing connection information will not be created, defaults to falseConnectionPool field connInfoSecretTargetDisabled, type boolean: When true, the secret containing connection information will not be created, defaults to falseGrafana field connInfoSecretTargetDisabled, type boolean: When true, the secret containing connection information will not be created, defaults to falseKafka field connInfoSecretTargetDisabled, type boolean: When true, the secret containing connection information will not be created, defaults to falseMySQL field connInfoSecretTargetDisabled, type boolean: When true, the secret containing connection information will not be created, defaults to falseOpenSearch field connInfoSecretTargetDisabled, type boolean: When true, the secret containing connection information will not be created, defaults to falsePostgreSQL field connInfoSecretTargetDisabled, type boolean: When true, the secret containing connection information will not be created, defaults to falseProject field connInfoSecretTargetDisabled, type boolean: When true, the secret containing connection information will not be created, defaults to falseRedis field connInfoSecretTargetDisabled, type boolean: When true, the secret containing connection information will not be created, defaults to falseServiceUser field connInfoSecretTargetDisabled, type boolean: When true, the secret containing connection information will not be created, defaults to falseKAFKA_SCHEMA_REGISTRY_URI and KAFKA_REST_URI to Kafka secretCONNECTIONPOOL_NAME in ConnectionPool secretCONNECTIONPOOL_PORT exposes service port instead of pool portSERVICEUSER_PORT when sasl is the only authentication methodPostgreSQL field userConfig.pg_qualstats.enabled: deprecatedPostgreSQL field userConfig.pg_qualstats.min_err_estimate_num: deprecatedPostgreSQL field userConfig.pg_qualstats.min_err_estimate_ratio: deprecatedPostgreSQL field userConfig.pg_qualstats.track_constants: deprecatedPostgreSQL field userConfig.pg_qualstats.track_pg_catalog: deprecatedPreconditions, CreateOrUpdate, Delete. Thanks to @ataraxKafka field userConfig.kafka.transaction_partition_verification_enable, type boolean: Enable verification that checks that the partition has been added to the transaction before writing transactional records to the partitionCassandra field userConfig.service_log, type boolean: Store logs for the service so that they are available in the HTTP API and consoleClickhouse field userConfig.service_log, type boolean: Store logs for the service so that they are available in the HTTP API and consoleGrafana field userConfig.service_log, type boolean: Store logs for the service so that they are available in the HTTP API and consoleKafkaConnect field userConfig.service_log, type boolean: Store logs for the service so that they are available in the HTTP API and consoleKafka field userConfig.kafka_rest_config.name_strategy_validation, type boolean: If true, validate that given schema is registered under expected subject name by the used name strategy when producing messagesKafka field userConfig.service_log, type boolean: Store logs for the service so that they are available in the HTTP API and consoleMySQL field userConfig.service_log, type boolean: Store logs for the service so that they are available in the HTTP API and consoleOpenSearch field userConfig.service_log, type boolean: Store logs for the service so that they are available in the HTTP API and consolePostgreSQL field userConfig.pg_qualstats, type object: System-wide settings for the pg_qualstats extensionPostgreSQL field userConfig.service_log, type boolean: Store logs for the service so that they are available in the HTTP API and consoleRedis field userConfig.service_log, type boolean: Store logs for the service so that they are available in the HTTP API and consoleServiceIntegration: do not send empty user config to the APIstring type fields to the documentationClickhouse field userConfig.private_access.clickhouse_mysql, type boolean: Allow clients to connect to clickhouse_mysql with a DNS name that always resolves to the service's private IP addressesClickhouse field userConfig.privatelink_access.clickhouse_mysql, type boolean: Enable clickhouse_mysqlClickhouse field userConfig.public_access.clickhouse_mysql, type boolean: Allow clients to connect to clickhouse_mysql from the public internet for service nodes that are in a project VPC or another type of private networkGrafana field userConfig.unified_alerting_enabled, type boolean: Enable or disable Grafana unified alerting functionalityKafka field userConfig.aiven_kafka_topic_messages, type boolean: Allow access to read Kafka topic messages in the Aiven Console and REST APIKafka field userConfig.kafka.sasl_oauthbearer_expected_audience, type string: The (optional) comma-delimited setting for the broker to use to verify that the JWT was issued for one of the expected audiencesKafka field userConfig.kafka.sasl_oauthbearer_expected_issuer, type string: Optional setting for the broker to use to verify that the JWT was created by the expected issuerKafka field userConfig.kafka.sasl_oauthbearer_jwks_endpoint_url, type string: OIDC JWKS endpoint URL. By setting this the SASL SSL OAuth2/OIDC authentication is enabledKafka field userConfig.kafka.sasl_oauthbearer_sub_claim_name, type string: Name of the scope from which to extract the subject claim from the JWT. Defaults to subKafka field userConfig.kafka_version: enum [3.1, 3.3, 3.4, 3.5] \u2192 [3.1, 3.3, 3.4, 3.5, 3.6]Kafka field userConfig.tiered_storage.local_cache.size: deprecatedOpenSearch field userConfig.opensearch.indices_memory_max_index_buffer_size, type integer: Absolute value. Default is unbound. Doesn't work without indices.memory.index_buffer_sizeOpenSearch field userConfig.opensearch.indices_memory_min_index_buffer_size, type integer: Absolute value. Default is 48mb. Doesn't work without indices.memory.index_buffer_sizeOpenSearch field userConfig.opensearch.auth_failure_listeners.internal_authentication_backend_limiting.authentication_backend: enum [internal]OpenSearch field userConfig.opensearch.auth_failure_listeners.internal_authentication_backend_limiting.type: enum [username]OpenSearch field userConfig.opensearch.auth_failure_listeners.ip_rate_limiting.type: enum [ip]OpenSearch field userConfig.opensearch.search_max_buckets: maximum 65536 \u2192 1000000ServiceIntegration field kafkaMirrormaker.kafka_mirrormaker.producer_max_request_size: maximum 67108864 \u2192 268435456projectVpcId and projectVPCRef mutablenil user config conversionCassandra kind option additional_backup_regionsGrafana kind option auto_loginKafka kind properties log_local_retention_bytes, log_local_retention_msKafka kind option remote_log_storage_system_enableOpenSearch kind option auth_failure_listenersOpenSearch kind Index State Management optionsKafkaKafka version 3.5Kafka spec property scheduled_rebalance_max_delay_msKafka spec property remote_log_storage_system_enableKafkaConnect spec property scheduled_rebalance_max_delay_msOpenSearch spec property openidKAFKA_SCHEMA_REGISTRY_HOST and KAFKA_SCHEMA_REGISTRY_PORT for KafkaKAFKA_CONNECT_HOST, KAFKA_CONNECT_PORT, KAFKA_REST_HOST and KAFKA_REST_PORT for Kafka. Thanks to @Dariuschunclean_leader_election_enable from KafkaTopic kind configKAFKA_SASL_PORT for Kafka kind if SASL authentication method is enabledredis options to datadog ServiceIntegrationCassandra version 3Kafka versions 3.1 and 3.4kafka_rest_config.producer_max_request_size optionkafka_mirrormaker.producer_compression_type optionclusterRole.create option to Helm chart. Thanks to @ryaneorthOpenSearch.spec.userConfig.idp_pemtrustedcas_content option. Specifies the PEM-encoded root certificate authority (CA) content for the SAML identity provider (IdP) server verification.ServiceIntegration kind SourceProjectName and DestinationProjectName fieldsServiceIntegration fields MaxLength validationServiceIntegration validation: multiple user configs cannot be setServiceIntegration, should not require destinationServiceName or sourceEndpointID fieldServiceIntegration, add missing external_aws_cloudwatch_metrics type config serializationServiceIntegration integration type listannotations and labels fields to connInfoSecretTargetOpenSearch.spec.userConfig.opensearch.search_max_buckets maximum to 65536plan as a required fieldminumim, maximum validations for number typeip_filter backward compatibilityclickhouseKafka.tables.data_format-property enum RawBLOB valueuserConfig.opensearch.email_sender_username validation patternlog_cleaner_min_cleanable_ratio minimum and maximum validation rules3.2, reached EOL10, reached EOLProjectVPC by ID to avoid conflictsProjectVPC deletion by exiting on DELETING statusClickhouseUser controllerClickhouseUser.spec.project and ClickhouseUser.spec.serviceName as immutablesignalfxAuthSecretRef fields marked as requireddatadog, kafka_connect, kafka_logs, metricsclickhouse_postgresql, clickhouse_kafka, clickhouse_kafka, logs, external_aws_cloudwatch_metricsKafkaTopic.Spec.topicName field. Unlike the metadata.name, supports additional characters and has a longer length. KafkaTopic.Spec.topicName replaces metadata.name in future releases and will be marked as required.false value for termination_protection propertymin_cleanable_dirty_ratio. Thanks to @TV2rdImportant: This release brings breaking changes to the userConfig property. After new charts are installed, update your existing instances manually using the kubectl edit command according to the API reference.
Note: It is now recommended to disable webhooks for Kubernetes version 1.25 and higher, as native CRD validation rules are used.
ip_filter field is now of object typeserviceIntegrations on service types. Only the read_replica type is available.min_cleanable_dirty_ratio config field supportspec.disk_space propertylinux/amd64 build. Thanks to @christoffer-eidenever from choices of maintenance dowdevelopment flag to configure logger's behaviormake generate-user-configs)genericServiceHandler to generalize service managementKafkaACL deletionProjectVPCRef property to Kafka, OpenSearch, Clickhouse and Redis kinds to get ProjectVPC ID when resource is readyProjectVPC deletion, deletes by ID first if possible, then tries by nameclient.Object storage update data lossfeatures:
improvements:
fixes:
improvements:
fixes:
features:
Use the following checks to help you troubleshoot the Aiven Operator for Kubernetes.
"},{"location":"troubleshooting.html#check-the-pods","title":"Check the Pods","text":"Verify that all the operator Pods are READY, and the STATUS is Running.
kubectl get pod -n aiven-operator-system\nThe output is similar to the following:
NAME READY STATUS RESTARTS AGE\naiven-operator-controller-manager-576d944499-ggttj 1/1 Running 0 12m\nVerify that the cert-manager Pods are also running.
kubectl get pod -n cert-manager\nThe output has the status:
NAME READY STATUS RESTARTS AGE\ncert-manager-7dd5854bb4-85cpv 1/1 Running 0 76s\ncert-manager-cainjector-64c949654c-n2z8l 1/1 Running 0 77s\ncert-manager-webhook-6bdffc7c9d-47w6z 1/1 Running 0 76s\nUse the following command to visualize all the logs from the operator.
kubectl logs -n aiven-operator-system -l control-plane=controller-manager\nkubectl get pod -n aiven-operator-system -l control-plane=controller-manager -o jsonpath=\"{.items[0].spec.containers[0].image}\"\nWe're always working to resolve problems that pop up in Aiven products. If your problem is listed below, we know about it and are working to fix it. If your problem isn't listed below, report it as an issue.
"},{"location":"troubleshooting.html#cert-manager","title":"cert-manager","text":""},{"location":"troubleshooting.html#issue","title":"Issue","text":"The following event appears on the operator Pod:
MountVolume.SetUp failed for volume \"cert\" : secret \"webhook-server-cert\" not found\nYou cannot run the operator.
"},{"location":"troubleshooting.html#solution","title":"Solution","text":"Make sure that cert-manager is up and running.
kubectl get pod -n cert-manager\nThe output shows the status of each cert-manager:
NAME READY STATUS RESTARTS AGE\ncert-manager-7dd5854bb4-85cpv 1/1 Running 0 76s\ncert-manager-cainjector-64c949654c-n2z8l 1/1 Running 0 77s\ncert-manager-webhook-6bdffc7c9d-47w6z 1/1 Running 0 76s\nAutogenerated from CRD files.
"},{"location":"api-reference/cassandra.html","title":"Cassandra","text":""},{"location":"api-reference/cassandra.html#usage-example","title":"Usage example","text":"ExampleapiVersion: aiven.io/v1alpha1\nkind: Cassandra\nmetadata:\n name: my-cassandra\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n connInfoSecretTarget:\n name: cassandra-secret\n prefix: MY_SECRET_PREFIX_\n annotations:\n foo: bar\n labels:\n baz: egg\n\n project: aiven-project-name\n cloudName: google-europe-west1\n plan: startup-4\n\n maintenanceWindowDow: sunday\n maintenanceWindowTime: 11:00:00\n\n userConfig:\n migrate_sstableloader: true\n public_access:\n prometheus: true\n ip_filter:\n - network: 0.0.0.0\n description: whatever\n - network: 10.20.0.0/16\nInfo
To create this resource, a Secret containing Aiven token must be created first.
Apply the resource with:
kubectl apply -f example.yaml\nVerify the newly created Cassandra:
kubectl get cassandras my-cassandra\nThe output is similar to the following:
Name Project Region Plan State \nmy-cassandra aiven-project-name google-europe-west1 startup-4 RUNNING \nTo view the details of the Secret, use the following command:
kubectl describe secret cassandra-secret\nYou can use the jq to quickly decode the Secret:
kubectl get secret cassandra-secret -o json | jq '.data | map_values(@base64d)'\nThe output is similar to the following:
{\n \"CASSANDRA_HOST\": \"<secret>\",\n \"CASSANDRA_PORT\": \"<secret>\",\n \"CASSANDRA_USER\": \"<secret>\",\n \"CASSANDRA_PASSWORD\": \"<secret>\",\n \"CASSANDRA_URI\": \"<secret>\",\n \"CASSANDRA_HOSTS\": \"<secret>\",\n \"CASSANDRA_CA_CERT\": \"<secret>\",\n}\nCassandra is the Schema for the cassandras API.
Exposes secret keys
CASSANDRA_HOST, CASSANDRA_PORT, CASSANDRA_USER, CASSANDRA_PASSWORD, CASSANDRA_URI, CASSANDRA_HOSTS, CASSANDRA_CA_CERT.
Required
apiVersion (string). Value aiven.io/v1alpha1.kind (string). Value Cassandra.metadata (object). Data that identifies the object, including a name string and optional namespace.spec (object). CassandraSpec defines the desired state of Cassandra. See below for nested schema.Appears on Cassandra.
CassandraSpec defines the desired state of Cassandra.
Required
plan (string, MaxLength: 128). Subscription plan.project (string, Immutable, Pattern: ^[a-zA-Z0-9_-]+$, MaxLength: 63). Identifies the project this resource belongs to.Optional
authSecretRef (object). Authentication reference to Aiven token in a secret. See below for nested schema.cloudName (string, MaxLength: 256). Cloud the service runs in.connInfoSecretTarget (object). Secret configuration. See below for nested schema.connInfoSecretTargetDisabled (boolean, Immutable). When true, the secret containing connection information will not be created, defaults to false. This field cannot be changed after resource creation.disk_space (string, Pattern: (?i)^[1-9][0-9]*(GiB|G)?$). The disk space of the service, possible values depend on the service type, the cloud provider and the project. Reducing will result in the service re-balancing. The removal of this field does not change the value.maintenanceWindowDow (string, Enum: monday, tuesday, wednesday, thursday, friday, saturday, sunday). Day of week when maintenance operations should be performed. One monday, tuesday, wednesday, etc.maintenanceWindowTime (string, MaxLength: 8). Time of day when maintenance operations should be performed. UTC time in HH:mm:ss format.projectVPCRef (object). ProjectVPCRef reference to ProjectVPC resource to use its ID as ProjectVPCID automatically. See below for nested schema.projectVpcId (string, MaxLength: 36). Identifier of the VPC the service should be in, if any.serviceIntegrations (array of objects, Immutable, MaxItems: 1). Service integrations to specify when creating a service. Not applied after initial service creation. See below for nested schema.tags (object, AdditionalProperties: string). Tags are key-value pairs that allow you to categorize services.technicalEmails (array of objects, MaxItems: 10). Defines the email addresses that will receive alerts about upcoming maintenance updates or warnings about service instability. See below for nested schema.terminationProtection (boolean). Prevent service from being deleted. It is recommended to have this enabled for all services.userConfig (object). Cassandra specific user configuration options. See below for nested schema.Appears on spec.
Authentication reference to Aiven token in a secret.
Required
key (string, MinLength: 1).name (string, MinLength: 1).Appears on spec.
Secret configuration.
Required
name (string, Immutable). Name of the secret resource to be created. By default, it is equal to the resource name.Optional
annotations (object, AdditionalProperties: string). Annotations added to the secret.labels (object, AdditionalProperties: string). Labels added to the secret.prefix (string). Prefix for the secret's keys. Added \"as is\" without any transformations. By default, is equal to the kind name in uppercase + underscore, e.g. KAFKA_, REDIS_, etc.Appears on spec.
ProjectVPCRef reference to ProjectVPC resource to use its ID as ProjectVPCID automatically.
Required
name (string, MinLength: 1).Optional
namespace (string, MinLength: 1).Appears on spec.
Service integrations to specify when creating a service. Not applied after initial service creation.
Required
integrationType (string, Enum: read_replica).sourceServiceName (string, MinLength: 1, MaxLength: 64).Appears on spec.
Defines the email addresses that will receive alerts about upcoming maintenance updates or warnings about service instability.
Required
email (string). Email address.Appears on spec.
Cassandra specific user configuration options.
Optional
additional_backup_regions (array of strings, MaxItems: 1). Deprecated. Additional Cloud Regions for Backup Replication.backup_hour (integer, Minimum: 0, Maximum: 23). The hour of day (in UTC) when backup for the service is started. New backup is only started if previous backup has already completed.backup_minute (integer, Minimum: 0, Maximum: 59). The minute of an hour when backup for the service is started. New backup is only started if previous backup has already completed.cassandra (object). cassandra configuration values. See below for nested schema.cassandra_version (string, Enum: 4, 4.1, Pattern: ^[0-9]+(\\.[0-9]+)?$). Cassandra version.ip_filter (array of objects, MaxItems: 1024). Allow incoming connections from CIDR address block, e.g. 10.20.0.0/16. See below for nested schema.migrate_sstableloader (boolean). Sets the service into migration mode enabling the sstableloader utility to be used to upload Cassandra data files. Available only on service create.private_access (object). Allow access to selected service ports from private networks. See below for nested schema.project_to_fork_from (string, Immutable, Pattern: ^[a-z][-a-z0-9]{0,63}$|^$, MaxLength: 63). Name of another project to fork a service from. This has effect only when a new service is being created.public_access (object). Allow access to selected service ports from the public Internet. See below for nested schema.service_log (boolean). Store logs for the service so that they are available in the HTTP API and console.service_to_fork_from (string, Immutable, Pattern: ^[a-z][-a-z0-9]{0,63}$|^$, MaxLength: 64). Name of another service to fork from. This has effect only when a new service is being created.service_to_join_with (string, Pattern: ^[a-z][-a-z0-9]{0,63}$, MaxLength: 64). When bootstrapping, instead of creating a new Cassandra cluster try to join an existing one from another service. Can only be set on service creation.static_ips (boolean). Use static public IP addresses.Appears on spec.userConfig.
cassandra configuration values.
Optional
batch_size_fail_threshold_in_kb (integer, Minimum: 1, Maximum: 1000000). Fail any multiple-partition batch exceeding this value. 50kb (10x warn threshold) by default.batch_size_warn_threshold_in_kb (integer, Minimum: 1, Maximum: 1000000). Log a warning message on any multiple-partition batch size exceeding this value.5kb per batch by default.Caution should be taken on increasing the size of this thresholdas it can lead to node instability.datacenter (string, MaxLength: 128). Name of the datacenter to which nodes of this service belong. Can be set only when creating the service.read_request_timeout_in_ms (integer, Minimum: 1000, Maximum: 10000). How long the coordinator waits for read operations to complete before timing it out. 5 seconds by default.write_request_timeout_in_ms (integer, Minimum: 1000, Maximum: 10000). How long the coordinator waits for write requests to complete with at least one node in the local datacenter. 2 seconds by default.Appears on spec.userConfig.
CIDR address block, either as a string, or in a dict with an optional description field.
Required
network (string, MaxLength: 43). CIDR address block.Optional
description (string, MaxLength: 1024). Description for IP filter list entry.Appears on spec.userConfig.
Allow access to selected service ports from private networks.
Required
prometheus (boolean). Allow clients to connect to prometheus with a DNS name that always resolves to the service's private IP addresses. Only available in certain network locations.Appears on spec.userConfig.
Allow access to selected service ports from the public Internet.
Required
prometheus (boolean). Allow clients to connect to prometheus from the public internet for service nodes that are in a project VPC or another type of private network.apiVersion: aiven.io/v1alpha1\nkind: Clickhouse\nmetadata:\n name: my-clickhouse\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n connInfoSecretTarget:\n name: my-clickhouse\n annotations:\n foo: bar\n labels:\n baz: egg\n\n tags:\n env: test\n instance: foo\n\n userConfig:\n ip_filter:\n - network: 0.0.0.0/32\n description: bar\n - network: 10.20.0.0/16\n\n project: my-aiven-project\n cloudName: google-europe-west1\n plan: startup-16\n\n maintenanceWindowDow: friday\n maintenanceWindowTime: 23:00:00\nInfo
To create this resource, a Secret containing Aiven token must be created first.
Apply the resource with:
kubectl apply -f example.yaml\nVerify the newly created Clickhouse:
kubectl get clickhouses my-clickhouse\nThe output is similar to the following:
Name Project Region Plan State \nmy-clickhouse my-aiven-project google-europe-west1 startup-16 RUNNING \nTo view the details of the Secret, use the following command:
kubectl describe secret my-clickhouse\nYou can use the jq to quickly decode the Secret:
kubectl get secret my-clickhouse -o json | jq '.data | map_values(@base64d)'\nThe output is similar to the following:
{\n \"CLICKHOUSE_HOST\": \"<secret>\",\n \"CLICKHOUSE_PORT\": \"<secret>\",\n \"CLICKHOUSE_USER\": \"<secret>\",\n \"CLICKHOUSE_PASSWORD\": \"<secret>\",\n}\nClickhouse is the Schema for the clickhouses API.
Exposes secret keys
CLICKHOUSE_HOST, CLICKHOUSE_PORT, CLICKHOUSE_USER, CLICKHOUSE_PASSWORD.
Required
apiVersion (string). Value aiven.io/v1alpha1.kind (string). Value Clickhouse.metadata (object). Data that identifies the object, including a name string and optional namespace.spec (object). ClickhouseSpec defines the desired state of Clickhouse. See below for nested schema.Appears on Clickhouse.
ClickhouseSpec defines the desired state of Clickhouse.
Required
plan (string, MaxLength: 128). Subscription plan.project (string, Immutable, Pattern: ^[a-zA-Z0-9_-]+$, MaxLength: 63). Identifies the project this resource belongs to.Optional
authSecretRef (object). Authentication reference to Aiven token in a secret. See below for nested schema.cloudName (string, MaxLength: 256). Cloud the service runs in.connInfoSecretTarget (object). Secret configuration. See below for nested schema.connInfoSecretTargetDisabled (boolean, Immutable). When true, the secret containing connection information will not be created, defaults to false. This field cannot be changed after resource creation.disk_space (string, Pattern: (?i)^[1-9][0-9]*(GiB|G)?$). The disk space of the service, possible values depend on the service type, the cloud provider and the project. Reducing will result in the service re-balancing. The removal of this field does not change the value.maintenanceWindowDow (string, Enum: monday, tuesday, wednesday, thursday, friday, saturday, sunday). Day of week when maintenance operations should be performed. One monday, tuesday, wednesday, etc.maintenanceWindowTime (string, MaxLength: 8). Time of day when maintenance operations should be performed. UTC time in HH:mm:ss format.projectVPCRef (object). ProjectVPCRef reference to ProjectVPC resource to use its ID as ProjectVPCID automatically. See below for nested schema.projectVpcId (string, MaxLength: 36). Identifier of the VPC the service should be in, if any.serviceIntegrations (array of objects, Immutable, MaxItems: 1). Service integrations to specify when creating a service. Not applied after initial service creation. See below for nested schema.tags (object, AdditionalProperties: string). Tags are key-value pairs that allow you to categorize services.technicalEmails (array of objects, MaxItems: 10). Defines the email addresses that will receive alerts about upcoming maintenance updates or warnings about service instability. See below for nested schema.terminationProtection (boolean). Prevent service from being deleted. It is recommended to have this enabled for all services.userConfig (object). OpenSearch specific user configuration options. See below for nested schema.Appears on spec.
Authentication reference to Aiven token in a secret.
Required
key (string, MinLength: 1).name (string, MinLength: 1).Appears on spec.
Secret configuration.
Required
name (string, Immutable). Name of the secret resource to be created. By default, it is equal to the resource name.Optional
annotations (object, AdditionalProperties: string). Annotations added to the secret.labels (object, AdditionalProperties: string). Labels added to the secret.prefix (string). Prefix for the secret's keys. Added \"as is\" without any transformations. By default, is equal to the kind name in uppercase + underscore, e.g. KAFKA_, REDIS_, etc.Appears on spec.
ProjectVPCRef reference to ProjectVPC resource to use its ID as ProjectVPCID automatically.
Required
name (string, MinLength: 1).Optional
namespace (string, MinLength: 1).Appears on spec.
Service integrations to specify when creating a service. Not applied after initial service creation.
Required
integrationType (string, Enum: read_replica).sourceServiceName (string, MinLength: 1, MaxLength: 64).Appears on spec.
Defines the email addresses that will receive alerts about upcoming maintenance updates or warnings about service instability.
Required
email (string). Email address.Appears on spec.
OpenSearch specific user configuration options.
Optional
additional_backup_regions (array of strings, MaxItems: 1). Deprecated. Additional Cloud Regions for Backup Replication.ip_filter (array of objects, MaxItems: 1024). Allow incoming connections from CIDR address block, e.g. 10.20.0.0/16. See below for nested schema.private_access (object). Allow access to selected service ports from private networks. See below for nested schema.privatelink_access (object). Allow access to selected service components through Privatelink. See below for nested schema.project_to_fork_from (string, Immutable, Pattern: ^[a-z][-a-z0-9]{0,63}$|^$, MaxLength: 63). Name of another project to fork a service from. This has effect only when a new service is being created.public_access (object). Allow access to selected service ports from the public Internet. See below for nested schema.service_log (boolean). Store logs for the service so that they are available in the HTTP API and console.service_to_fork_from (string, Immutable, Pattern: ^[a-z][-a-z0-9]{0,63}$|^$, MaxLength: 64). Name of another service to fork from. This has effect only when a new service is being created.static_ips (boolean). Use static public IP addresses.Appears on spec.userConfig.
CIDR address block, either as a string, or in a dict with an optional description field.
Required
network (string, MaxLength: 43). CIDR address block.Optional
description (string, MaxLength: 1024). Description for IP filter list entry.Appears on spec.userConfig.
Allow access to selected service ports from private networks.
Optional
clickhouse (boolean). Allow clients to connect to clickhouse with a DNS name that always resolves to the service's private IP addresses. Only available in certain network locations.clickhouse_https (boolean). Allow clients to connect to clickhouse_https with a DNS name that always resolves to the service's private IP addresses. Only available in certain network locations.clickhouse_mysql (boolean). Allow clients to connect to clickhouse_mysql with a DNS name that always resolves to the service's private IP addresses. Only available in certain network locations.prometheus (boolean). Allow clients to connect to prometheus with a DNS name that always resolves to the service's private IP addresses. Only available in certain network locations.Appears on spec.userConfig.
Allow access to selected service components through Privatelink.
Optional
clickhouse (boolean). Enable clickhouse.clickhouse_https (boolean). Enable clickhouse_https.clickhouse_mysql (boolean). Enable clickhouse_mysql.prometheus (boolean). Enable prometheus.Appears on spec.userConfig.
Allow access to selected service ports from the public Internet.
Optional
clickhouse (boolean). Allow clients to connect to clickhouse from the public internet for service nodes that are in a project VPC or another type of private network.clickhouse_https (boolean). Allow clients to connect to clickhouse_https from the public internet for service nodes that are in a project VPC or another type of private network.clickhouse_mysql (boolean). Allow clients to connect to clickhouse_mysql from the public internet for service nodes that are in a project VPC or another type of private network.prometheus (boolean). Allow clients to connect to prometheus from the public internet for service nodes that are in a project VPC or another type of private network.apiVersion: aiven.io/v1alpha1\nkind: ClickhouseDatabase\nmetadata:\n name: my-db\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n project: my-aiven-project\n serviceName: my-clickhouse\n databaseName: example-db\nInfo
To create this resource, a Secret containing Aiven token must be created first.
Apply the resource with:
kubectl apply -f example.yaml\nVerify the newly created ClickhouseDatabase:
kubectl get clickhousedatabases my-db\nThe output is similar to the following:
Name Database name Service Name Project \nmy-db example-db my-clickhouse my-aiven-project \nClickhouseDatabase is the Schema for the databases API.
Required
apiVersion (string). Value aiven.io/v1alpha1.kind (string). Value ClickhouseDatabase.metadata (object). Data that identifies the object, including a name string and optional namespace.spec (object). ClickhouseDatabaseSpec defines the desired state of ClickhouseDatabase. See below for nested schema.Appears on ClickhouseDatabase.
ClickhouseDatabaseSpec defines the desired state of ClickhouseDatabase.
Required
project (string, Immutable, Pattern: ^[a-zA-Z0-9_-]+$, MaxLength: 63). Identifies the project this resource belongs to.serviceName (string, Immutable, Pattern: ^[a-z][-a-z0-9]+$, MaxLength: 63). Specifies the name of the service that this resource belongs to.Optional
authSecretRef (object). Authentication reference to Aiven token in a secret. See below for nested schema.databaseName (string, Immutable, MaxLength: 63). Specifies the Clickhouse database name. Defaults to metadata.name if omitted.Note
metadata.name is ASCII-only. For UTF-8 names, use spec.databaseName, but ASCII is advised for compatibility.
Appears on spec.
Authentication reference to Aiven token in a secret.
Required
key (string, MinLength: 1).name (string, MinLength: 1).apiVersion: aiven.io/v1alpha1\nkind: ClickhouseGrant\nmetadata:\n name: demo-ch-grant\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n project: my-aiven-project\n serviceName: my-clickhouse\n\n privilegeGrants:\n - grantees:\n - user: user1\n - user: my-clickhouse-user-\ud83e\udd84\n privileges:\n - SELECT\n - INSERT\n database: my-db\n # If table is omitted, the privileges are granted on all tables in the database\n # If columns is omitted, the privileges are granted on all columns in the table\n - grantees:\n - role: my-role\n privileges:\n - SELECT\n database: my-db\n table: my-table\n columns:\n - col1\n - col2\n\n roleGrants:\n - roles:\n - other-role\n grantees:\n - user: my-user\n - role: my-role\napiVersion: aiven.io/v1alpha1\nkind: ClickhouseGrant\nmetadata:\n name: my-clickhouse-grant\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n project: aiven-project-name\n serviceName: my-clickhouse-service\n\n privilegeGrants:\n - grantees:\n - role: my-clickhouse-role\n privileges:\n - INSERT\n - SELECT\n - CREATE TABLE\n - CREATE VIEW\n database: my-clickhouse-db\n roleGrants:\n - grantees:\n - user: my-clickhouse-user\n roles:\n - my-clickhouse-role\n\n---\n\napiVersion: aiven.io/v1alpha1\nkind: Clickhouse\nmetadata:\n name: my-clickhouse-service\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n project: aiven-project-name\n cloudName: google-europe-west1\n plan: startup-16\n\n---\n\napiVersion: aiven.io/v1alpha1\nkind: ClickhouseDatabase\nmetadata:\n name: my-clickhouse-db\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n project: aiven-project-name\n serviceName: my-clickhouse-service\n\n---\n\napiVersion: aiven.io/v1alpha1\nkind: ClickhouseUser\nmetadata:\n name: my-clickhouse-user\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n project: aiven-project-name\n serviceName: my-clickhouse-service\n\n---\n\napiVersion: aiven.io/v1alpha1\nkind: ClickhouseRole\nmetadata:\n name: my-clickhouse-role\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n project: aiven-project-name\n serviceName: my-clickhouse-service\n role: my-clickhouse-role\nInfo
To create this resource, a Secret containing Aiven token must be created first.
Apply the resource with:
kubectl apply -f example.yaml\nVerify the newly created ClickhouseGrant:
kubectl get clickhousegrants demo-ch-grant\nThe output is similar to the following:
Name Project Service Name \ndemo-ch-grant my-aiven-project my-clickhouse \nClickhouseGrant is the Schema for the ClickhouseGrants API
Warning
Due to the way ClickHouse operates, updating this resource first revokes the existing privileges.
Required
apiVersion (string). Value aiven.io/v1alpha1.kind (string). Value ClickhouseGrant.metadata (object). Data that identifies the object, including a name string and optional namespace.spec (object). ClickhouseGrantSpec defines the desired state of ClickhouseGrant. See below for nested schema.Appears on ClickhouseGrant.
ClickhouseGrantSpec defines the desired state of ClickhouseGrant.
Required
project (string, Immutable, Pattern: ^[a-zA-Z0-9_-]+$, MaxLength: 63). Identifies the project this resource belongs to.serviceName (string, Immutable, Pattern: ^[a-z][-a-z0-9]+$, MaxLength: 63). Specifies the name of the service that this resource belongs to.Optional
authSecretRef (object). Authentication reference to Aiven token in a secret. See below for nested schema.privilegeGrants (array of objects). Configuration to grant a privilege. Privileges not in the manifest are revoked. Existing privileges are retained; new ones are granted. See below for nested schema.roleGrants (array of objects). Configuration to grant a role. Role grants not in the manifest are revoked. Existing role grants are retained; new ones are granted. See below for nested schema.Appears on spec.
Authentication reference to Aiven token in a secret.
Required
key (string, MinLength: 1).name (string, MinLength: 1).Appears on spec.
PrivilegeGrant represents the privileges to be granted to users or roles. See.
Required
database (string). The database that the grant refers to.grantees (array of objects, MinItems: 1). List of grantees (users or roles) to grant the privilege to. See below for nested schema.privileges (array of strings). The privileges to grant, i.e. INSERT, SELECT. See.Optional
columns (array of strings). The column that the grant refers to.table (string). The tables that the grant refers to. To grant a privilege on all tables in a database, omit this field instead of writing table: \"*\".withGrantOption (boolean). If true, then the grantee (user or role) get the permission to execute the GRANT query. Users can grant privileges of the same scope they have and less. See.Appears on spec.privilegeGrants.
Grantee represents a user or a role to which privileges or roles are granted.
Ambiguity in the GRANT syntax
Due to an ambiguity in the GRANT syntax in ClickHouse, users and roles should not share the same name. It is unclear whether a grant applies to the user or the role.
Optional
role (string).user (string).Appears on spec.
RoleGrant represents the roles to be assigned to users or roles. See.
Required
grantees (array of objects, MinItems: 1). List of grantees (users or roles) to grant the privilege to. See below for nested schema.roles (array of strings, MinItems: 1). List of roles to grant to the grantees.Optional
withAdminOption (boolean). If true, the grant is executed with ADMIN OPTION privilege. See.Appears on spec.roleGrants.
Grantee represents a user or a role to which privileges or roles are granted.
Ambiguity in the GRANT syntax
Due to an ambiguity in the GRANT syntax in ClickHouse, users and roles should not share the same name. It is unclear whether a grant applies to the user or the role.
Optional
role (string).user (string).apiVersion: aiven.io/v1alpha1\nkind: ClickhouseRole\nmetadata:\n name: my-role\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n project: my-aiven-project\n serviceName: my-clickhouse\n role: my-role\nInfo
To create this resource, a Secret containing Aiven token must be created first.
Apply the resource with:
kubectl apply -f example.yaml\nVerify the newly created ClickhouseRole:
kubectl get clickhouseroles my-role\nThe output is similar to the following:
Name Project Service Name Role \nmy-role my-aiven-project my-clickhouse my-role \nClickhouseRole is the Schema for the clickhouseroles API.
Required
apiVersion (string). Value aiven.io/v1alpha1.kind (string). Value ClickhouseRole.metadata (object). Data that identifies the object, including a name string and optional namespace.spec (object). ClickhouseRoleSpec defines the desired state of ClickhouseRole. See below for nested schema.Appears on ClickhouseRole.
ClickhouseRoleSpec defines the desired state of ClickhouseRole.
Required
project (string, Immutable, Pattern: ^[a-zA-Z0-9_-]+$, MaxLength: 63). Identifies the project this resource belongs to.role (string, Immutable, MaxLength: 255). The role that is to be created.serviceName (string, Immutable, Pattern: ^[a-z][-a-z0-9]+$, MaxLength: 63). Specifies the name of the service that this resource belongs to.Optional
authSecretRef (object). Authentication reference to Aiven token in a secret. See below for nested schema.Appears on spec.
Authentication reference to Aiven token in a secret.
Required
key (string, MinLength: 1).name (string, MinLength: 1).apiVersion: aiven.io/v1alpha1\nkind: ClickhouseUser\nmetadata:\n name: my-clickhouse-user\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n connInfoSecretTarget:\n name: clickhouse-user-secret\n annotations:\n foo: bar\n labels:\n baz: egg\n\n project: my-aiven-project\n serviceName: my-clickhouse\n username: example-username\n\n---\n\napiVersion: aiven.io/v1alpha1\nkind: Clickhouse\nmetadata:\n name: my-clickhouse\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n project: my-aiven-project\n cloudName: google-europe-west1\n plan: startup-16\nInfo
To create this resource, a Secret containing Aiven token must be created first.
Apply the resource with:
kubectl apply -f example.yaml\nVerify the newly created ClickhouseUser:
kubectl get clickhouseusers my-clickhouse-user\nThe output is similar to the following:
Name Username Service Name Project \nmy-clickhouse-user example-username my-clickhouse my-aiven-project \nTo view the details of the Secret, use the following command:
kubectl describe secret clickhouse-user-secret\nYou can use the jq to quickly decode the Secret:
kubectl get secret clickhouse-user-secret -o json | jq '.data | map_values(@base64d)'\nThe output is similar to the following:
{\n \"CLICKHOUSEUSER_HOST\": \"<secret>\",\n \"CLICKHOUSEUSER_PORT\": \"<secret>\",\n \"CLICKHOUSEUSER_USER\": \"<secret>\",\n \"CLICKHOUSEUSER_PASSWORD\": \"<secret>\",\n}\nClickhouseUser is the Schema for the clickhouseusers API.
Exposes secret keys
CLICKHOUSEUSER_HOST, CLICKHOUSEUSER_PORT, CLICKHOUSEUSER_USER, CLICKHOUSEUSER_PASSWORD.
Required
apiVersion (string). Value aiven.io/v1alpha1.kind (string). Value ClickhouseUser.metadata (object). Data that identifies the object, including a name string and optional namespace.spec (object). ClickhouseUserSpec defines the desired state of ClickhouseUser. See below for nested schema.Appears on ClickhouseUser.
ClickhouseUserSpec defines the desired state of ClickhouseUser.
Required
project (string, Immutable, Pattern: ^[a-zA-Z0-9_-]+$, MaxLength: 63). Identifies the project this resource belongs to.serviceName (string, Immutable, Pattern: ^[a-z][-a-z0-9]+$, MaxLength: 63). Specifies the name of the service that this resource belongs to.Optional
authSecretRef (object). Authentication reference to Aiven token in a secret. See below for nested schema.connInfoSecretTarget (object). Secret configuration. See below for nested schema.connInfoSecretTargetDisabled (boolean, Immutable). When true, the secret containing connection information will not be created, defaults to false. This field cannot be changed after resource creation.username (string, Immutable, MaxLength: 63). Name of the Clickhouse user. Defaults to metadata.name if omitted.Note
metadata.name is ASCII-only. For UTF-8 names, use spec.username, but ASCII is advised for compatibility.
Appears on spec.
Authentication reference to Aiven token in a secret.
Required
key (string, MinLength: 1).name (string, MinLength: 1).Appears on spec.
Secret configuration.
Required
name (string, Immutable). Name of the secret resource to be created. By default, it is equal to the resource name.Optional
annotations (object, AdditionalProperties: string). Annotations added to the secret.labels (object, AdditionalProperties: string). Labels added to the secret.prefix (string). Prefix for the secret's keys. Added \"as is\" without any transformations. By default, is equal to the kind name in uppercase + underscore, e.g. KAFKA_, REDIS_, etc.apiVersion: aiven.io/v1alpha1\nkind: ConnectionPool\nmetadata:\n name: my-connection-pool\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n project: aiven-project-name\n serviceName: my-pg\n databaseName: my-database\n username: my-service-user\n poolMode: transaction\n poolSize: 25\n\n---\n\napiVersion: aiven.io/v1alpha1\nkind: PostgreSQL\nmetadata:\n name: my-pg\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n project: aiven-project-name\n cloudName: google-europe-west1\n plan: startup-4\n\n---\n\napiVersion: aiven.io/v1alpha1\nkind: Database\nmetadata:\n name: my-database\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n project: aiven-project-name\n serviceName: my-pg\n\n---\n\napiVersion: aiven.io/v1alpha1\nkind: ServiceUser\nmetadata:\n name: my-service-user\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n project: aiven-project-name\n serviceName: my-pg\nInfo
To create this resource, a Secret containing Aiven token must be created first.
Apply the resource with:
kubectl apply -f example.yaml\nVerify the newly created ConnectionPool:
kubectl get connectionpools my-connection-pool\nThe output is similar to the following:
Name Service Name Project Database Username Pool Size Pool Mode \nmy-connection-pool my-pg aiven-project-name my-database my-service-user 25 transaction \nConnectionPool is the Schema for the connectionpools API.
Exposes secret keys
CONNECTIONPOOL_NAME, CONNECTIONPOOL_HOST, CONNECTIONPOOL_PORT, CONNECTIONPOOL_DATABASE, CONNECTIONPOOL_USER, CONNECTIONPOOL_PASSWORD, CONNECTIONPOOL_SSLMODE, CONNECTIONPOOL_DATABASE_URI, CONNECTIONPOOL_CA_CERT.
Required
apiVersion (string). Value aiven.io/v1alpha1.kind (string). Value ConnectionPool.metadata (object). Data that identifies the object, including a name string and optional namespace.spec (object). ConnectionPoolSpec defines the desired state of ConnectionPool. See below for nested schema.Appears on ConnectionPool.
ConnectionPoolSpec defines the desired state of ConnectionPool.
Required
databaseName (string, MaxLength: 40). Name of the database the pool connects to.project (string, Immutable, Pattern: ^[a-zA-Z0-9_-]+$, MaxLength: 63). Identifies the project this resource belongs to.serviceName (string, Immutable, Pattern: ^[a-z][-a-z0-9]+$, MaxLength: 63). Specifies the name of the service that this resource belongs to.username (string, MaxLength: 64). Name of the service user used to connect to the database.Optional
authSecretRef (object). Authentication reference to Aiven token in a secret. See below for nested schema.connInfoSecretTarget (object). Secret configuration. See below for nested schema.connInfoSecretTargetDisabled (boolean, Immutable). When true, the secret containing connection information will not be created, defaults to false. This field cannot be changed after resource creation.poolMode (string, Enum: session, transaction, statement). Mode the pool operates in (session, transaction, statement).poolSize (integer). Number of connections the pool may create towards the backend server.Appears on spec.
Authentication reference to Aiven token in a secret.
Required
key (string, MinLength: 1).name (string, MinLength: 1).Appears on spec.
Secret configuration.
Required
name (string, Immutable). Name of the secret resource to be created. By default, it is equal to the resource name.Optional
annotations (object, AdditionalProperties: string). Annotations added to the secret.labels (object, AdditionalProperties: string). Labels added to the secret.prefix (string). Prefix for the secret's keys. Added \"as is\" without any transformations. By default, is equal to the kind name in uppercase + underscore, e.g. KAFKA_, REDIS_, etc.apiVersion: aiven.io/v1alpha1\nkind: Database\nmetadata:\n name: my-db\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n project: aiven-project-name\n serviceName: my-service\n\n lcCtype: en_US.UTF-8\n lcCollate: en_US.UTF-8\nInfo
To create this resource, a Secret containing Aiven token must be created first.
Apply the resource with:
kubectl apply -f example.yaml\nVerify the newly created Database:
kubectl get databases my-db\nThe output is similar to the following:
Name Project Service Name \nmy-db aiven-project-name my-service \nDatabase is the Schema for the databases API.
Required
apiVersion (string). Value aiven.io/v1alpha1.kind (string). Value Database.metadata (object). Data that identifies the object, including a name string and optional namespace.spec (object). DatabaseSpec defines the desired state of Database. See below for nested schema.Appears on Database.
DatabaseSpec defines the desired state of Database.
Required
project (string, Immutable, Pattern: ^[a-zA-Z0-9_-]+$, MaxLength: 63). Identifies the project this resource belongs to.serviceName (string, Immutable, Pattern: ^[a-z][-a-z0-9]+$, MaxLength: 63). Specifies the name of the service that this resource belongs to.Optional
authSecretRef (object). Authentication reference to Aiven token in a secret. See below for nested schema.lcCollate (string, Immutable, MaxLength: 128). Default string sort order (LC_COLLATE) of the database. Default value: en_US.UTF-8.lcCtype (string, Immutable, MaxLength: 128). Default character classification (LC_CTYPE) of the database. Default value: en_US.UTF-8.terminationProtection (boolean). It is a Kubernetes side deletion protections, which prevents the database from being deleted by Kubernetes. It is recommended to enable this for any production databases containing critical data.Appears on spec.
Authentication reference to Aiven token in a secret.
Required
key (string, MinLength: 1).name (string, MinLength: 1).apiVersion: aiven.io/v1alpha1\nkind: Grafana\nmetadata:\n name: my-grafana\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n connInfoSecretTarget:\n name: grafana-secret\n prefix: MY_SECRET_PREFIX_\n annotations:\n foo: bar\n labels:\n baz: egg\n\n project: my-aiven-project\n cloudName: google-europe-west1\n plan: startup-1\n\n maintenanceWindowDow: sunday\n maintenanceWindowTime: 11:00:00\n\n userConfig:\n public_access:\n grafana: true\n ip_filter:\n - network: 0.0.0.0\n description: whatever\n - network: 10.20.0.0/16\nInfo
To create this resource, a Secret containing Aiven token must be created first.
Apply the resource with:
kubectl apply -f example.yaml\nVerify the newly created Grafana:
kubectl get grafanas my-grafana\nThe output is similar to the following:
Name Project Region Plan State \nmy-grafana my-aiven-project google-europe-west1 startup-1 RUNNING \nTo view the details of the Secret, use the following command:
kubectl describe secret grafana-secret\nYou can use the jq to quickly decode the Secret:
kubectl get secret grafana-secret -o json | jq '.data | map_values(@base64d)'\nThe output is similar to the following:
{\n \"GRAFANA_HOST\": \"<secret>\",\n \"GRAFANA_PORT\": \"<secret>\",\n \"GRAFANA_USER\": \"<secret>\",\n \"GRAFANA_PASSWORD\": \"<secret>\",\n \"GRAFANA_URI\": \"<secret>\",\n \"GRAFANA_HOSTS\": \"<secret>\",\n}\nGrafana is the Schema for the grafanas API.
Exposes secret keys
GRAFANA_HOST, GRAFANA_PORT, GRAFANA_USER, GRAFANA_PASSWORD, GRAFANA_URI, GRAFANA_HOSTS.
Required
apiVersion (string). Value aiven.io/v1alpha1.kind (string). Value Grafana.metadata (object). Data that identifies the object, including a name string and optional namespace.spec (object). GrafanaSpec defines the desired state of Grafana. See below for nested schema.Appears on Grafana.
GrafanaSpec defines the desired state of Grafana.
Required
plan (string, MaxLength: 128). Subscription plan.project (string, Immutable, Pattern: ^[a-zA-Z0-9_-]+$, MaxLength: 63). Identifies the project this resource belongs to.Optional
authSecretRef (object). Authentication reference to Aiven token in a secret. See below for nested schema.cloudName (string, MaxLength: 256). Cloud the service runs in.connInfoSecretTarget (object). Secret configuration. See below for nested schema.connInfoSecretTargetDisabled (boolean, Immutable). When true, the secret containing connection information will not be created, defaults to false. This field cannot be changed after resource creation.disk_space (string, Pattern: (?i)^[1-9][0-9]*(GiB|G)?$). The disk space of the service, possible values depend on the service type, the cloud provider and the project. Reducing will result in the service re-balancing. The removal of this field does not change the value.maintenanceWindowDow (string, Enum: monday, tuesday, wednesday, thursday, friday, saturday, sunday). Day of week when maintenance operations should be performed. One monday, tuesday, wednesday, etc.maintenanceWindowTime (string, MaxLength: 8). Time of day when maintenance operations should be performed. UTC time in HH:mm:ss format.projectVPCRef (object). ProjectVPCRef reference to ProjectVPC resource to use its ID as ProjectVPCID automatically. See below for nested schema.projectVpcId (string, MaxLength: 36). Identifier of the VPC the service should be in, if any.serviceIntegrations (array of objects, Immutable, MaxItems: 1). Service integrations to specify when creating a service. Not applied after initial service creation. See below for nested schema.tags (object, AdditionalProperties: string). Tags are key-value pairs that allow you to categorize services.technicalEmails (array of objects, MaxItems: 10). Defines the email addresses that will receive alerts about upcoming maintenance updates or warnings about service instability. See below for nested schema.terminationProtection (boolean). Prevent service from being deleted. It is recommended to have this enabled for all services.userConfig (object). Cassandra specific user configuration options. See below for nested schema.Appears on spec.
Authentication reference to Aiven token in a secret.
Required
key (string, MinLength: 1).name (string, MinLength: 1).Appears on spec.
Secret configuration.
Required
name (string, Immutable). Name of the secret resource to be created. By default, it is equal to the resource name.Optional
annotations (object, AdditionalProperties: string). Annotations added to the secret.labels (object, AdditionalProperties: string). Labels added to the secret.prefix (string). Prefix for the secret's keys. Added \"as is\" without any transformations. By default, is equal to the kind name in uppercase + underscore, e.g. KAFKA_, REDIS_, etc.Appears on spec.
ProjectVPCRef reference to ProjectVPC resource to use its ID as ProjectVPCID automatically.
Required
name (string, MinLength: 1).Optional
namespace (string, MinLength: 1).Appears on spec.
Service integrations to specify when creating a service. Not applied after initial service creation.
Required
integrationType (string, Enum: read_replica).sourceServiceName (string, MinLength: 1, MaxLength: 64).Appears on spec.
Defines the email addresses that will receive alerts about upcoming maintenance updates or warnings about service instability.
Required
email (string). Email address.Appears on spec.
Cassandra specific user configuration options.
Optional
additional_backup_regions (array of strings, MaxItems: 1). Additional Cloud Regions for Backup Replication.alerting_enabled (boolean). Enable or disable Grafana legacy alerting functionality. This should not be enabled with unified_alerting_enabled.alerting_error_or_timeout (string, Enum: alerting, keep_state). Default error or timeout setting for new alerting rules.alerting_max_annotations_to_keep (integer, Minimum: 0, Maximum: 1000000). Max number of alert annotations that Grafana stores. 0 (default) keeps all alert annotations.alerting_nodata_or_nullvalues (string, Enum: alerting, no_data, keep_state, ok). Default value for 'no data or null values' for new alerting rules.allow_embedding (boolean). Allow embedding Grafana dashboards with iframe/frame/object/embed tags. Disabled by default to limit impact of clickjacking.auth_azuread (object). Azure AD OAuth integration. See below for nested schema.auth_basic_enabled (boolean). Enable or disable basic authentication form, used by Grafana built-in login.auth_generic_oauth (object). Generic OAuth integration. See below for nested schema.auth_github (object). Github Auth integration. See below for nested schema.auth_gitlab (object). GitLab Auth integration. See below for nested schema.auth_google (object). Google Auth integration. See below for nested schema.cookie_samesite (string, Enum: lax, strict, none). Cookie SameSite attribute: strict prevents sending cookie for cross-site requests, effectively disabling direct linking from other sites to Grafana. lax is the default value.custom_domain (string, MaxLength: 255). Serve the web frontend using a custom CNAME pointing to the Aiven DNS name.dashboard_previews_enabled (boolean). This feature is new in Grafana 9 and is quite resource intensive. It may cause low-end plans to work more slowly while the dashboard previews are rendering.dashboards_min_refresh_interval (string, Pattern: ^[0-9]+(ms|s|m|h|d)$, MaxLength: 16). Signed sequence of decimal numbers, followed by a unit suffix (ms, s, m, h, d), e.g. 30s, 1h.dashboards_versions_to_keep (integer, Minimum: 1, Maximum: 100). Dashboard versions to keep per dashboard.dataproxy_send_user_header (boolean). Send X-Grafana-User header to data source.dataproxy_timeout (integer, Minimum: 15, Maximum: 90). Timeout for data proxy requests in seconds.date_formats (object). Grafana date format specifications. See below for nested schema.disable_gravatar (boolean). Set to true to disable gravatar. Defaults to false (gravatar is enabled).editors_can_admin (boolean). Editors can manage folders, teams and dashboards created by them.external_image_storage (object). External image store settings. See below for nested schema.google_analytics_ua_id (string, Pattern: ^(G|UA|YT|MO)-[a-zA-Z0-9-]+$, MaxLength: 64). Google Analytics ID.ip_filter (array of objects, MaxItems: 1024). Allow incoming connections from CIDR address block, e.g. 10.20.0.0/16. See below for nested schema.metrics_enabled (boolean). Enable Grafana /metrics endpoint.oauth_allow_insecure_email_lookup (boolean). Enforce user lookup based on email instead of the unique ID provided by the IdP.private_access (object). Allow access to selected service ports from private networks. See below for nested schema.privatelink_access (object). Allow access to selected service components through Privatelink. See below for nested schema.project_to_fork_from (string, Immutable, Pattern: ^[a-z][-a-z0-9]{0,63}$|^$, MaxLength: 63). Name of another project to fork a service from. This has effect only when a new service is being created.public_access (object). Allow access to selected service ports from the public Internet. See below for nested schema.recovery_basebackup_name (string, Pattern: ^[a-zA-Z0-9-_:.]+$, MaxLength: 128). Name of the basebackup to restore in forked service.service_log (boolean). Store logs for the service so that they are available in the HTTP API and console.service_to_fork_from (string, Immutable, Pattern: ^[a-z][-a-z0-9]{0,63}$|^$, MaxLength: 64). Name of another service to fork from. This has effect only when a new service is being created.smtp_server (object). SMTP server settings. See below for nested schema.static_ips (boolean). Use static public IP addresses.unified_alerting_enabled (boolean). Enable or disable Grafana unified alerting functionality. By default this is enabled and any legacy alerts will be migrated on upgrade to Grafana 9+. To stay on legacy alerting, set unified_alerting_enabled to false and alerting_enabled to true. See https://grafana.com/docs/grafana/latest/alerting/set-up/migrating-alerts/ for more details.user_auto_assign_org (boolean). Auto-assign new users on signup to main organization. Defaults to false.user_auto_assign_org_role (string, Enum: Viewer, Admin, Editor). Set role for new signups. Defaults to Viewer.viewers_can_edit (boolean). Users with view-only permission can edit but not save dashboards.Appears on spec.userConfig.
Azure AD OAuth integration.
Required
auth_url (string, MaxLength: 2048). Authorization URL.client_id (string, Pattern: ^[\\040-\\176]+$, MaxLength: 1024). Client ID from provider.client_secret (string, Pattern: ^[\\040-\\176]+$, MaxLength: 1024). Client secret from provider.token_url (string, MaxLength: 2048). Token URL.Optional
allow_sign_up (boolean). Automatically sign-up users on successful sign-in.allowed_domains (array of strings, MaxItems: 50). Allowed domains.allowed_groups (array of strings, MaxItems: 50). Require users to belong to one of given groups.Appears on spec.userConfig.
Generic OAuth integration.
Required
api_url (string, MaxLength: 2048). API URL.auth_url (string, MaxLength: 2048). Authorization URL.client_id (string, Pattern: ^[\\040-\\176]+$, MaxLength: 1024). Client ID from provider.client_secret (string, Pattern: ^[\\040-\\176]+$, MaxLength: 1024). Client secret from provider.token_url (string, MaxLength: 2048). Token URL.Optional
allow_sign_up (boolean). Automatically sign-up users on successful sign-in.allowed_domains (array of strings, MaxItems: 50). Allowed domains.allowed_organizations (array of strings, MaxItems: 50). Require user to be member of one of the listed organizations.auto_login (boolean). Allow users to bypass the login screen and automatically log in.name (string, Pattern: ^[a-zA-Z0-9_\\- ]+$, MaxLength: 128). Name of the OAuth integration.scopes (array of strings, MaxItems: 50). OAuth scopes.Appears on spec.userConfig.
Github Auth integration.
Required
client_id (string, Pattern: ^[\\040-\\176]+$, MaxLength: 1024). Client ID from provider.client_secret (string, Pattern: ^[\\040-\\176]+$, MaxLength: 1024). Client secret from provider.Optional
allow_sign_up (boolean). Automatically sign-up users on successful sign-in.allowed_organizations (array of strings, MaxItems: 50). Require users to belong to one of given organizations.auto_login (boolean). Allow users to bypass the login screen and automatically log in.skip_org_role_sync (boolean). Stop automatically syncing user roles.team_ids (array of integers, MaxItems: 50). Require users to belong to one of given team IDs.Appears on spec.userConfig.
GitLab Auth integration.
Required
allowed_groups (array of strings, MaxItems: 50). Require users to belong to one of given groups.client_id (string, Pattern: ^[\\040-\\176]+$, MaxLength: 1024). Client ID from provider.client_secret (string, Pattern: ^[\\040-\\176]+$, MaxLength: 1024). Client secret from provider.Optional
allow_sign_up (boolean). Automatically sign-up users on successful sign-in.api_url (string, MaxLength: 2048). API URL. This only needs to be set when using self hosted GitLab.auth_url (string, MaxLength: 2048). Authorization URL. This only needs to be set when using self hosted GitLab.token_url (string, MaxLength: 2048). Token URL. This only needs to be set when using self hosted GitLab.Appears on spec.userConfig.
Google Auth integration.
Required
allowed_domains (array of strings, MaxItems: 64). Domains allowed to sign-in to this Grafana.client_id (string, Pattern: ^[\\040-\\176]+$, MaxLength: 1024). Client ID from provider.client_secret (string, Pattern: ^[\\040-\\176]+$, MaxLength: 1024). Client secret from provider.Optional
allow_sign_up (boolean). Automatically sign-up users on successful sign-in.Appears on spec.userConfig.
Grafana date format specifications.
Optional
default_timezone (string, MaxLength: 64). Default time zone for user preferences. Value browser uses browser local time zone.full_date (string, MaxLength: 128). Moment.js style format string for cases where full date is shown.interval_day (string, MaxLength: 128). Moment.js style format string used when a time requiring day accuracy is shown.interval_hour (string, MaxLength: 128). Moment.js style format string used when a time requiring hour accuracy is shown.interval_minute (string, MaxLength: 128). Moment.js style format string used when a time requiring minute accuracy is shown.interval_month (string, MaxLength: 128). Moment.js style format string used when a time requiring month accuracy is shown.interval_second (string, MaxLength: 128). Moment.js style format string used when a time requiring second accuracy is shown.interval_year (string, MaxLength: 128). Moment.js style format string used when a time requiring year accuracy is shown.Appears on spec.userConfig.
External image store settings.
Required
access_key (string, Pattern: ^[A-Z0-9]+$, MaxLength: 4096). S3 access key. Requires permissions to the S3 bucket for the s3:PutObject and s3:PutObjectAcl actions.bucket_url (string, MaxLength: 2048). Bucket URL for S3.provider (string, Enum: s3). Provider type.secret_key (string, Pattern: ^[A-Za-z0-9/+=]+$, MaxLength: 4096). S3 secret key.Appears on spec.userConfig.
CIDR address block, either as a string, or in a dict with an optional description field.
Required
network (string, MaxLength: 43). CIDR address block.Optional
description (string, MaxLength: 1024). Description for IP filter list entry.Appears on spec.userConfig.
Allow access to selected service ports from private networks.
Required
grafana (boolean). Allow clients to connect to grafana with a DNS name that always resolves to the service's private IP addresses. Only available in certain network locations.Appears on spec.userConfig.
Allow access to selected service components through Privatelink.
Required
grafana (boolean). Enable grafana.Appears on spec.userConfig.
Allow access to selected service ports from the public Internet.
Required
grafana (boolean). Allow clients to connect to grafana from the public internet for service nodes that are in a project VPC or another type of private network.Appears on spec.userConfig.
SMTP server settings.
Required
from_address (string, MaxLength: 319). Address used for sending emails.host (string, MaxLength: 255). Server hostname or IP.port (integer, Minimum: 1, Maximum: 65535). SMTP server port.Optional
from_name (string, Pattern: ^[^\\x00-\\x1F]+$, MaxLength: 128). Name used in outgoing emails, defaults to Grafana.password (string, Pattern: ^[^\\x00-\\x1F]+$, MaxLength: 255). Password for SMTP authentication.skip_verify (boolean). Skip verifying server certificate. Defaults to false.starttls_policy (string, Enum: OpportunisticStartTLS, MandatoryStartTLS, NoStartTLS). Either OpportunisticStartTLS, MandatoryStartTLS or NoStartTLS. Default is OpportunisticStartTLS.username (string, Pattern: ^[^\\x00-\\x1F]+$, MaxLength: 255). Username for SMTP authentication.apiVersion: aiven.io/v1alpha1\nkind: Kafka\nmetadata:\n name: my-kafka\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n connInfoSecretTarget:\n name: kafka-secret\n prefix: MY_SECRET_PREFIX_\n annotations:\n foo: bar\n labels:\n baz: egg\n\n project: my-aiven-project\n cloudName: google-europe-west1\n plan: startup-2\n\n maintenanceWindowDow: friday\n maintenanceWindowTime: 23:00:00\nInfo
To create this resource, a Secret containing Aiven token must be created first.
Apply the resource with:
kubectl apply -f example.yaml\nVerify the newly created Kafka:
kubectl get kafkas my-kafka\nThe output is similar to the following:
Name Project Region Plan State \nmy-kafka my-aiven-project google-europe-west1 startup-2 RUNNING \nTo view the details of the Secret, use the following command:
kubectl describe secret kafka-secret\nYou can use the jq to quickly decode the Secret:
kubectl get secret kafka-secret -o json | jq '.data | map_values(@base64d)'\nThe output is similar to the following:
{\n \"KAFKA_HOST\": \"<secret>\",\n \"KAFKA_PORT\": \"<secret>\",\n \"KAFKA_USERNAME\": \"<secret>\",\n \"KAFKA_PASSWORD\": \"<secret>\",\n \"KAFKA_ACCESS_CERT\": \"<secret>\",\n \"KAFKA_ACCESS_KEY\": \"<secret>\",\n \"KAFKA_SASL_HOST\": \"<secret>\",\n \"KAFKA_SASL_PORT\": \"<secret>\",\n \"KAFKA_SCHEMA_REGISTRY_HOST\": \"<secret>\",\n \"KAFKA_SCHEMA_REGISTRY_PORT\": \"<secret>\",\n \"KAFKA_CONNECT_HOST\": \"<secret>\",\n \"KAFKA_CONNECT_PORT\": \"<secret>\",\n \"KAFKA_REST_HOST\": \"<secret>\",\n \"KAFKA_REST_PORT\": \"<secret>\",\n \"KAFKA_CA_CERT\": \"<secret>\",\n}\nKafka is the Schema for the kafkas API.
Exposes secret keys
KAFKA_HOST, KAFKA_PORT, KAFKA_USERNAME, KAFKA_PASSWORD, KAFKA_ACCESS_CERT, KAFKA_ACCESS_KEY, KAFKA_SASL_HOST, KAFKA_SASL_PORT, KAFKA_SCHEMA_REGISTRY_HOST, KAFKA_SCHEMA_REGISTRY_PORT, KAFKA_CONNECT_HOST, KAFKA_CONNECT_PORT, KAFKA_REST_HOST, KAFKA_REST_PORT, KAFKA_CA_CERT.
Required
apiVersion (string). Value aiven.io/v1alpha1.kind (string). Value Kafka.metadata (object). Data that identifies the object, including a name string and optional namespace.spec (object). KafkaSpec defines the desired state of Kafka. See below for nested schema.Appears on Kafka.
KafkaSpec defines the desired state of Kafka.
Required
plan (string, MaxLength: 128). Subscription plan.project (string, Immutable, Pattern: ^[a-zA-Z0-9_-]+$, MaxLength: 63). Identifies the project this resource belongs to.Optional
authSecretRef (object). Authentication reference to Aiven token in a secret. See below for nested schema.cloudName (string, MaxLength: 256). Cloud the service runs in.connInfoSecretTarget (object). Secret configuration. See below for nested schema.connInfoSecretTargetDisabled (boolean, Immutable). When true, the secret containing connection information will not be created, defaults to false. This field cannot be changed after resource creation.disk_space (string, Pattern: (?i)^[1-9][0-9]*(GiB|G)?$). The disk space of the service, possible values depend on the service type, the cloud provider and the project. Reducing will result in the service re-balancing. The removal of this field does not change the value.karapace (boolean). Switch the service to use Karapace for schema registry and REST proxy.maintenanceWindowDow (string, Enum: monday, tuesday, wednesday, thursday, friday, saturday, sunday). Day of week when maintenance operations should be performed. One monday, tuesday, wednesday, etc.maintenanceWindowTime (string, MaxLength: 8). Time of day when maintenance operations should be performed. UTC time in HH:mm:ss format.projectVPCRef (object). ProjectVPCRef reference to ProjectVPC resource to use its ID as ProjectVPCID automatically. See below for nested schema.projectVpcId (string, MaxLength: 36). Identifier of the VPC the service should be in, if any.serviceIntegrations (array of objects, Immutable, MaxItems: 1). Service integrations to specify when creating a service. Not applied after initial service creation. See below for nested schema.tags (object, AdditionalProperties: string). Tags are key-value pairs that allow you to categorize services.technicalEmails (array of objects, MaxItems: 10). Defines the email addresses that will receive alerts about upcoming maintenance updates or warnings about service instability. See below for nested schema.terminationProtection (boolean). Prevent service from being deleted. It is recommended to have this enabled for all services.userConfig (object). Kafka specific user configuration options. See below for nested schema.Appears on spec.
Authentication reference to Aiven token in a secret.
Required
key (string, MinLength: 1).name (string, MinLength: 1).Appears on spec.
Secret configuration.
Required
name (string, Immutable). Name of the secret resource to be created. By default, it is equal to the resource name.Optional
annotations (object, AdditionalProperties: string). Annotations added to the secret.labels (object, AdditionalProperties: string). Labels added to the secret.prefix (string). Prefix for the secret's keys. Added \"as is\" without any transformations. By default, is equal to the kind name in uppercase + underscore, e.g. KAFKA_, REDIS_, etc.Appears on spec.
ProjectVPCRef reference to ProjectVPC resource to use its ID as ProjectVPCID automatically.
Required
name (string, MinLength: 1).Optional
namespace (string, MinLength: 1).Appears on spec.
Service integrations to specify when creating a service. Not applied after initial service creation.
Required
integrationType (string, Enum: read_replica).sourceServiceName (string, MinLength: 1, MaxLength: 64).Appears on spec.
Defines the email addresses that will receive alerts about upcoming maintenance updates or warnings about service instability.
Required
email (string). Email address.Appears on spec.
Kafka specific user configuration options.
Optional
additional_backup_regions (array of strings, MaxItems: 1). Deprecated. Additional Cloud Regions for Backup Replication.aiven_kafka_topic_messages (boolean). Allow access to read Kafka topic messages in the Aiven Console and REST API.custom_domain (string, MaxLength: 255). Serve the web frontend using a custom CNAME pointing to the Aiven DNS name.ip_filter (array of objects, MaxItems: 1024). Allow incoming connections from CIDR address block, e.g. 10.20.0.0/16. See below for nested schema.kafka (object). Kafka broker configuration values. See below for nested schema.kafka_authentication_methods (object). Kafka authentication methods. See below for nested schema.kafka_connect (boolean). Enable Kafka Connect service.kafka_connect_config (object). Kafka Connect configuration values. See below for nested schema.kafka_connect_secret_providers. See below for } (array of objects). Configure external secret providers in order to reference external secrets in connector configuration. Currently Hashicorp Vault (provider: vault, auth_method: token) and AWS Secrets Manager (provider: aws, auth_method: credentials) are supported. Secrets can be referenced in connector config with ${::nested schema. kafka_rest (boolean). Enable Kafka-REST service.kafka_rest_authorization (boolean). Enable authorization in Kafka-REST service.kafka_rest_config (object). Kafka REST configuration. See below for nested schema.kafka_version (string, Enum: 3.4, 3.5, 3.6, 3.7). Kafka major version.letsencrypt_sasl_privatelink (boolean). Use Letsencrypt CA for Kafka SASL via Privatelink.private_access (object). Allow access to selected service ports from private networks. See below for nested schema.privatelink_access (object). Allow access to selected service components through Privatelink. See below for nested schema.public_access (object). Allow access to selected service ports from the public Internet. See below for nested schema.schema_registry (boolean). Enable Schema-Registry service.schema_registry_config (object). Schema Registry configuration. See below for nested schema.service_log (boolean). Store logs for the service so that they are available in the HTTP API and console.static_ips (boolean). Use static public IP addresses.tiered_storage (object). Tiered storage configuration. See below for nested schema.Appears on spec.userConfig.
CIDR address block, either as a string, or in a dict with an optional description field.
Required
network (string, MaxLength: 43). CIDR address block.Optional
description (string, MaxLength: 1024). Description for IP filter list entry.Appears on spec.userConfig.
Kafka broker configuration values.
Optional
auto_create_topics_enable (boolean). Enable auto creation of topics.compression_type (string, Enum: gzip, snappy, lz4, zstd, uncompressed, producer). Specify the final compression type for a given topic. This configuration accepts the standard compression codecs (gzip, snappy, lz4, zstd). It additionally accepts uncompressed which is equivalent to no compression; and producer which means retain the original compression codec set by the producer.connections_max_idle_ms (integer, Minimum: 1000, Maximum: 3600000). Idle connections timeout: the server socket processor threads close the connections that idle for longer than this.default_replication_factor (integer, Minimum: 1, Maximum: 10). Replication factor for autocreated topics.group_initial_rebalance_delay_ms (integer, Minimum: 0, Maximum: 300000). The amount of time, in milliseconds, the group coordinator will wait for more consumers to join a new group before performing the first rebalance. A longer delay means potentially fewer rebalances, but increases the time until processing begins. The default value for this is 3 seconds. During development and testing it might be desirable to set this to 0 in order to not delay test execution time.group_max_session_timeout_ms (integer, Minimum: 0, Maximum: 1800000). The maximum allowed session timeout for registered consumers. Longer timeouts give consumers more time to process messages in between heartbeats at the cost of a longer time to detect failures.group_min_session_timeout_ms (integer, Minimum: 0, Maximum: 60000). The minimum allowed session timeout for registered consumers. Longer timeouts give consumers more time to process messages in between heartbeats at the cost of a longer time to detect failures.log_cleaner_delete_retention_ms (integer, Minimum: 0, Maximum: 315569260000). How long are delete records retained?.log_cleaner_max_compaction_lag_ms (integer, Minimum: 30000). The maximum amount of time message will remain uncompacted. Only applicable for logs that are being compacted.log_cleaner_min_cleanable_ratio (number, Minimum: 0.2, Maximum: 0.9). Controls log compactor frequency. Larger value means more frequent compactions but also more space wasted for logs. Consider setting log.cleaner.max.compaction.lag.ms to enforce compactions sooner, instead of setting a very high value for this option.log_cleaner_min_compaction_lag_ms (integer, Minimum: 0). The minimum time a message will remain uncompacted in the log. Only applicable for logs that are being compacted.log_cleanup_policy (string, Enum: delete, compact, compact,delete). The default cleanup policy for segments beyond the retention window.log_flush_interval_messages (integer, Minimum: 1). The number of messages accumulated on a log partition before messages are flushed to disk.log_flush_interval_ms (integer, Minimum: 0). The maximum time in ms that a message in any topic is kept in memory before flushed to disk. If not set, the value in log.flush.scheduler.interval.ms is used.log_index_interval_bytes (integer, Minimum: 0, Maximum: 104857600). The interval with which Kafka adds an entry to the offset index.log_index_size_max_bytes (integer, Minimum: 1048576, Maximum: 104857600). The maximum size in bytes of the offset index.log_local_retention_bytes (integer, Minimum: -2). The maximum size of local log segments that can grow for a partition before it gets eligible for deletion. If set to -2, the value of log.retention.bytes is used. The effective value should always be less than or equal to log.retention.bytes value.log_local_retention_ms (integer, Minimum: -2). The number of milliseconds to keep the local log segments before it gets eligible for deletion. If set to -2, the value of log.retention.ms is used. The effective value should always be less than or equal to log.retention.ms value.log_message_downconversion_enable (boolean). This configuration controls whether down-conversion of message formats is enabled to satisfy consume requests.log_message_timestamp_difference_max_ms (integer, Minimum: 0). The maximum difference allowed between the timestamp when a broker receives a message and the timestamp specified in the message.log_message_timestamp_type (string, Enum: CreateTime, LogAppendTime). Define whether the timestamp in the message is message create time or log append time.log_preallocate (boolean). Should pre allocate file when create new segment?.log_retention_bytes (integer, Minimum: -1). The maximum size of the log before deleting messages.log_retention_hours (integer, Minimum: -1, Maximum: 2147483647). The number of hours to keep a log file before deleting it.log_retention_ms (integer, Minimum: -1). The number of milliseconds to keep a log file before deleting it (in milliseconds), If not set, the value in log.retention.minutes is used. If set to -1, no time limit is applied.log_roll_jitter_ms (integer, Minimum: 0). The maximum jitter to subtract from logRollTimeMillis (in milliseconds). If not set, the value in log.roll.jitter.hours is used.log_roll_ms (integer, Minimum: 1). The maximum time before a new log segment is rolled out (in milliseconds).log_segment_bytes (integer, Minimum: 10485760, Maximum: 1073741824). The maximum size of a single log file.log_segment_delete_delay_ms (integer, Minimum: 0, Maximum: 3600000). The amount of time to wait before deleting a file from the filesystem.max_connections_per_ip (integer, Minimum: 256, Maximum: 2147483647). The maximum number of connections allowed from each ip address (defaults to 2147483647).max_incremental_fetch_session_cache_slots (integer, Minimum: 1000, Maximum: 10000). The maximum number of incremental fetch sessions that the broker will maintain.message_max_bytes (integer, Minimum: 0, Maximum: 100001200). The maximum size of message that the server can receive.min_insync_replicas (integer, Minimum: 1, Maximum: 7). When a producer sets acks to all (or -1), min.insync.replicas specifies the minimum number of replicas that must acknowledge a write for the write to be considered successful.num_partitions (integer, Minimum: 1, Maximum: 1000). Number of partitions for autocreated topics.offsets_retention_minutes (integer, Minimum: 1, Maximum: 2147483647). Log retention window in minutes for offsets topic.producer_purgatory_purge_interval_requests (integer, Minimum: 10, Maximum: 10000). The purge interval (in number of requests) of the producer request purgatory(defaults to 1000).replica_fetch_max_bytes (integer, Minimum: 1048576, Maximum: 104857600). The number of bytes of messages to attempt to fetch for each partition (defaults to 1048576). This is not an absolute maximum, if the first record batch in the first non-empty partition of the fetch is larger than this value, the record batch will still be returned to ensure that progress can be made.replica_fetch_response_max_bytes (integer, Minimum: 10485760, Maximum: 1048576000). Maximum bytes expected for the entire fetch response (defaults to 10485760). Records are fetched in batches, and if the first record batch in the first non-empty partition of the fetch is larger than this value, the record batch will still be returned to ensure that progress can be made. As such, this is not an absolute maximum.sasl_oauthbearer_expected_audience (string, Pattern: ^[^\\r\\n]*$, MaxLength: 128). The (optional) comma-delimited setting for the broker to use to verify that the JWT was issued for one of the expected audiences.sasl_oauthbearer_expected_issuer (string, Pattern: ^[^\\r\\n]*$, MaxLength: 128). Optional setting for the broker to use to verify that the JWT was created by the expected issuer.sasl_oauthbearer_jwks_endpoint_url (string, MaxLength: 2048). OIDC JWKS endpoint URL. By setting this the SASL SSL OAuth2/OIDC authentication is enabled. See also other options for SASL OAuth2/OIDC.sasl_oauthbearer_sub_claim_name (string, Pattern: ^[^\\r\\n]*$, MaxLength: 128). Name of the scope from which to extract the subject claim from the JWT. Defaults to sub.socket_request_max_bytes (integer, Minimum: 10485760, Maximum: 209715200). The maximum number of bytes in a socket request (defaults to 104857600).transaction_partition_verification_enable (boolean). Enable verification that checks that the partition has been added to the transaction before writing transactional records to the partition.transaction_remove_expired_transaction_cleanup_interval_ms (integer, Minimum: 600000, Maximum: 3600000). The interval at which to remove transactions that have expired due to transactional.id.expiration.ms passing (defaults to 3600000 (1 hour)).transaction_state_log_segment_bytes (integer, Minimum: 1048576, Maximum: 2147483647). The transaction topic segment bytes should be kept relatively small in order to facilitate faster log compaction and cache loads (defaults to 104857600 (100 mebibytes)).Appears on spec.userConfig.
Kafka authentication methods.
Optional
certificate (boolean). Enable certificate/SSL authentication.sasl (boolean). Enable SASL authentication.Appears on spec.userConfig.
Kafka Connect configuration values.
Optional
connector_client_config_override_policy (string, Enum: None, All). Defines what client configurations can be overridden by the connector. Default is None.consumer_auto_offset_reset (string, Enum: earliest, latest). What to do when there is no initial offset in Kafka or if the current offset does not exist any more on the server. Default is earliest.consumer_fetch_max_bytes (integer, Minimum: 1048576, Maximum: 104857600). Records are fetched in batches by the consumer, and if the first record batch in the first non-empty partition of the fetch is larger than this value, the record batch will still be returned to ensure that the consumer can make progress. As such, this is not a absolute maximum.consumer_isolation_level (string, Enum: read_uncommitted, read_committed). Transaction read isolation level. read_uncommitted is the default, but read_committed can be used if consume-exactly-once behavior is desired.consumer_max_partition_fetch_bytes (integer, Minimum: 1048576, Maximum: 104857600). Records are fetched in batches by the consumer.If the first record batch in the first non-empty partition of the fetch is larger than this limit, the batch will still be returned to ensure that the consumer can make progress.consumer_max_poll_interval_ms (integer, Minimum: 1, Maximum: 2147483647). The maximum delay in milliseconds between invocations of poll() when using consumer group management (defaults to 300000).consumer_max_poll_records (integer, Minimum: 1, Maximum: 10000). The maximum number of records returned in a single call to poll() (defaults to 500).offset_flush_interval_ms (integer, Minimum: 1, Maximum: 100000000). The interval at which to try committing offsets for tasks (defaults to 60000).offset_flush_timeout_ms (integer, Minimum: 1, Maximum: 2147483647). Maximum number of milliseconds to wait for records to flush and partition offset data to be committed to offset storage before cancelling the process and restoring the offset data to be committed in a future attempt (defaults to 5000).producer_batch_size (integer, Minimum: 0, Maximum: 5242880). This setting gives the upper bound of the batch size to be sent. If there are fewer than this many bytes accumulated for this partition, the producer will linger for the linger.ms time waiting for more records to show up. A batch size of zero will disable batching entirely (defaults to 16384).producer_buffer_memory (integer, Minimum: 5242880, Maximum: 134217728). The total bytes of memory the producer can use to buffer records waiting to be sent to the broker (defaults to 33554432).producer_compression_type (string, Enum: gzip, snappy, lz4, zstd, none). Specify the default compression type for producers. This configuration accepts the standard compression codecs (gzip, snappy, lz4, zstd). It additionally accepts none which is the default and equivalent to no compression.producer_linger_ms (integer, Minimum: 0, Maximum: 5000). This setting gives the upper bound on the delay for batching: once there is batch.size worth of records for a partition it will be sent immediately regardless of this setting, however if there are fewer than this many bytes accumulated for this partition the producer will linger for the specified time waiting for more records to show up. Defaults to 0.producer_max_request_size (integer, Minimum: 131072, Maximum: 67108864). This setting will limit the number of record batches the producer will send in a single request to avoid sending huge requests.scheduled_rebalance_max_delay_ms (integer, Minimum: 0, Maximum: 600000). The maximum delay that is scheduled in order to wait for the return of one or more departed workers before rebalancing and reassigning their connectors and tasks to the group. During this period the connectors and tasks of the departed workers remain unassigned. Defaults to 5 minutes.session_timeout_ms (integer, Minimum: 1, Maximum: 2147483647). The timeout in milliseconds used to detect failures when using Kafka\u2019s group management facilities (defaults to 10000).Appears on spec.userConfig.
SecretProvider.
Required
name (string). Name of the secret provider. Used to reference secrets in connector config.Optional
aws (object). AWS config for Secret Provider. See below for nested schema.vault (object). Vault Config for Secret Provider. See below for nested schema.Appears on spec.userConfig.kafka_connect_secret_providers.
AWS config for Secret Provider.
Required
auth_method (string, Enum: credentials). Auth method of the vault secret provider.region (string, MaxLength: 64). Region used to lookup secrets with AWS SecretManager.Optional
access_key (string, MaxLength: 128). Access key used to authenticate with aws.secret_key (string, MaxLength: 128). Secret key used to authenticate with aws.Appears on spec.userConfig.kafka_connect_secret_providers.
Vault Config for Secret Provider.
Required
address (string, MinLength: 1, MaxLength: 65536). Address of the Vault server.auth_method (string, Enum: token). Auth method of the vault secret provider.Optional
engine_version (integer, Enum: 1, 2). KV Secrets Engine version of the Vault server instance.token (string, MaxLength: 256). Token used to authenticate with vault and auth method token.Appears on spec.userConfig.
Kafka REST configuration.
Optional
consumer_enable_auto_commit (boolean). If true the consumer's offset will be periodically committed to Kafka in the background.consumer_request_max_bytes (integer, Minimum: 0, Maximum: 671088640). Maximum number of bytes in unencoded message keys and values by a single request.consumer_request_timeout_ms (integer, Enum: 1000, 15000, 30000, Minimum: 1000, Maximum: 30000). The maximum total time to wait for messages for a request if the maximum number of messages has not yet been reached.name_strategy (string, Enum: topic_name, record_name, topic_record_name). Name strategy to use when selecting subject for storing schemas.name_strategy_validation (boolean). If true, validate that given schema is registered under expected subject name by the used name strategy when producing messages.producer_acks (string, Enum: all, -1, 0, 1). The number of acknowledgments the producer requires the leader to have received before considering a request complete. If set to all or -1, the leader will wait for the full set of in-sync replicas to acknowledge the record.producer_compression_type (string, Enum: gzip, snappy, lz4, zstd, none). Specify the default compression type for producers. This configuration accepts the standard compression codecs (gzip, snappy, lz4, zstd). It additionally accepts none which is the default and equivalent to no compression.producer_linger_ms (integer, Minimum: 0, Maximum: 5000). Wait for up to the given delay to allow batching records together.producer_max_request_size (integer, Minimum: 0, Maximum: 2147483647). The maximum size of a request in bytes. Note that Kafka broker can also cap the record batch size.simpleconsumer_pool_size_max (integer, Minimum: 10, Maximum: 250). Maximum number of SimpleConsumers that can be instantiated per broker.Appears on spec.userConfig.
Allow access to selected service ports from private networks.
Optional
kafka (boolean). Allow clients to connect to kafka with a DNS name that always resolves to the service's private IP addresses. Only available in certain network locations.kafka_connect (boolean). Allow clients to connect to kafka_connect with a DNS name that always resolves to the service's private IP addresses. Only available in certain network locations.kafka_rest (boolean). Allow clients to connect to kafka_rest with a DNS name that always resolves to the service's private IP addresses. Only available in certain network locations.prometheus (boolean). Allow clients to connect to prometheus with a DNS name that always resolves to the service's private IP addresses. Only available in certain network locations.schema_registry (boolean). Allow clients to connect to schema_registry with a DNS name that always resolves to the service's private IP addresses. Only available in certain network locations.Appears on spec.userConfig.
Allow access to selected service components through Privatelink.
Optional
jolokia (boolean). Enable jolokia.kafka (boolean). Enable kafka.kafka_connect (boolean). Enable kafka_connect.kafka_rest (boolean). Enable kafka_rest.prometheus (boolean). Enable prometheus.schema_registry (boolean). Enable schema_registry.Appears on spec.userConfig.
Allow access to selected service ports from the public Internet.
Optional
kafka (boolean). Allow clients to connect to kafka from the public internet for service nodes that are in a project VPC or another type of private network.kafka_connect (boolean). Allow clients to connect to kafka_connect from the public internet for service nodes that are in a project VPC or another type of private network.kafka_rest (boolean). Allow clients to connect to kafka_rest from the public internet for service nodes that are in a project VPC or another type of private network.prometheus (boolean). Allow clients to connect to prometheus from the public internet for service nodes that are in a project VPC or another type of private network.schema_registry (boolean). Allow clients to connect to schema_registry from the public internet for service nodes that are in a project VPC or another type of private network.Appears on spec.userConfig.
Schema Registry configuration.
Optional
leader_eligibility (boolean). If true, Karapace / Schema Registry on the service nodes can participate in leader election. It might be needed to disable this when the schemas topic is replicated to a secondary cluster and Karapace / Schema Registry there must not participate in leader election. Defaults to true.topic_name (string, MinLength: 1, MaxLength: 249). The durable single partition topic that acts as the durable log for the data. This topic must be compacted to avoid losing data due to retention policy. Please note that changing this configuration in an existing Schema Registry / Karapace setup leads to previous schemas being inaccessible, data encoded with them potentially unreadable and schema ID sequence put out of order. It's only possible to do the switch while Schema Registry / Karapace is disabled. Defaults to _schemas.Appears on spec.userConfig.
Tiered storage configuration.
Optional
enabled (boolean). Whether to enable the tiered storage functionality.local_cache (object). Deprecated. Local cache configuration. See below for nested schema.Appears on spec.userConfig.tiered_storage.
Deprecated. Local cache configuration.
Required
size (integer, Minimum: 1, Maximum: 107374182400). Deprecated. Local cache size in bytes.apiVersion: aiven.io/v1alpha1\nkind: KafkaACL\nmetadata:\n name: my-kafka-acl\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n project: my-aiven-project\n serviceName: my-kafka\n topic: my-topic\n username: my-user\n permission: admin\nInfo
To create this resource, a Secret containing Aiven token must be created first.
Apply the resource with:
kubectl apply -f example.yaml\nVerify the newly created KafkaACL:
kubectl get kafkaacls my-kafka-acl\nThe output is similar to the following:
Name Service Name Project Username Permission Topic \nmy-kafka-acl my-kafka my-aiven-project my-user admin my-topic \nKafkaACL is the Schema for the kafkaacls API.
Required
apiVersion (string). Value aiven.io/v1alpha1.kind (string). Value KafkaACL.metadata (object). Data that identifies the object, including a name string and optional namespace.spec (object). KafkaACLSpec defines the desired state of KafkaACL. See below for nested schema.Appears on KafkaACL.
KafkaACLSpec defines the desired state of KafkaACL.
Required
permission (string, Enum: admin, read, readwrite, write). Kafka permission to grant (admin, read, readwrite, write).project (string, Immutable, Pattern: ^[a-zA-Z0-9_-]+$, MaxLength: 63). Identifies the project this resource belongs to.serviceName (string, Immutable, Pattern: ^[a-z][-a-z0-9]+$, MaxLength: 63). Specifies the name of the service that this resource belongs to.topic (string). Topic name pattern for the ACL entry.username (string). Username pattern for the ACL entry.Optional
authSecretRef (object). Authentication reference to Aiven token in a secret. See below for nested schema.Appears on spec.
Authentication reference to Aiven token in a secret.
Required
key (string, MinLength: 1).name (string, MinLength: 1).apiVersion: aiven.io/v1alpha1\nkind: KafkaConnect\nmetadata:\n name: my-kafka-connect\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n project: my-aiven-project\n cloudName: google-europe-west1\n plan: business-4\n\n userConfig:\n kafka_connect:\n consumer_isolation_level: read_committed\n public_access:\n kafka_connect: true\nInfo
To create this resource, a Secret containing Aiven token must be created first.
Apply the resource with:
kubectl apply -f example.yaml\nVerify the newly created KafkaConnect:
kubectl get kafkaconnects my-kafka-connect\nThe output is similar to the following:
Name Project Region Plan State \nmy-kafka-connect my-aiven-project google-europe-west1 business-4 RUNNING \nKafkaConnect is the Schema for the kafkaconnects API.
Required
apiVersion (string). Value aiven.io/v1alpha1.kind (string). Value KafkaConnect.metadata (object). Data that identifies the object, including a name string and optional namespace.spec (object). KafkaConnectSpec defines the desired state of KafkaConnect. See below for nested schema.Appears on KafkaConnect.
KafkaConnectSpec defines the desired state of KafkaConnect.
Required
plan (string, MaxLength: 128). Subscription plan.project (string, Immutable, Pattern: ^[a-zA-Z0-9_-]+$, MaxLength: 63). Identifies the project this resource belongs to.Optional
authSecretRef (object). Authentication reference to Aiven token in a secret. See below for nested schema.cloudName (string, MaxLength: 256). Cloud the service runs in.maintenanceWindowDow (string, Enum: monday, tuesday, wednesday, thursday, friday, saturday, sunday). Day of week when maintenance operations should be performed. One monday, tuesday, wednesday, etc.maintenanceWindowTime (string, MaxLength: 8). Time of day when maintenance operations should be performed. UTC time in HH:mm:ss format.projectVPCRef (object). ProjectVPCRef reference to ProjectVPC resource to use its ID as ProjectVPCID automatically. See below for nested schema.projectVpcId (string, MaxLength: 36). Identifier of the VPC the service should be in, if any.serviceIntegrations (array of objects, Immutable, MaxItems: 1). Service integrations to specify when creating a service. Not applied after initial service creation. See below for nested schema.tags (object, AdditionalProperties: string). Tags are key-value pairs that allow you to categorize services.technicalEmails (array of objects, MaxItems: 10). Defines the email addresses that will receive alerts about upcoming maintenance updates or warnings about service instability. See below for nested schema.terminationProtection (boolean). Prevent service from being deleted. It is recommended to have this enabled for all services.userConfig (object). KafkaConnect specific user configuration options. See below for nested schema.Appears on spec.
Authentication reference to Aiven token in a secret.
Required
key (string, MinLength: 1).name (string, MinLength: 1).Appears on spec.
ProjectVPCRef reference to ProjectVPC resource to use its ID as ProjectVPCID automatically.
Required
name (string, MinLength: 1).Optional
namespace (string, MinLength: 1).Appears on spec.
Service integrations to specify when creating a service. Not applied after initial service creation.
Required
integrationType (string, Enum: read_replica).sourceServiceName (string, MinLength: 1, MaxLength: 64).Appears on spec.
Defines the email addresses that will receive alerts about upcoming maintenance updates or warnings about service instability.
Required
email (string). Email address.Appears on spec.
KafkaConnect specific user configuration options.
Optional
additional_backup_regions (array of strings, MaxItems: 1). Deprecated. Additional Cloud Regions for Backup Replication.ip_filter (array of objects, MaxItems: 1024). Allow incoming connections from CIDR address block, e.g. 10.20.0.0/16. See below for nested schema.kafka_connect (object). Kafka Connect configuration values. See below for nested schema.private_access (object). Allow access to selected service ports from private networks. See below for nested schema.privatelink_access (object). Allow access to selected service components through Privatelink. See below for nested schema.public_access (object). Allow access to selected service ports from the public Internet. See below for nested schema.secret_providers. See below for } (array of objects). Configure external secret providers in order to reference external secrets in connector configuration. Currently Hashicorp Vault (provider: vault, auth_method: token) and AWS Secrets Manager (provider: aws, auth_method: credentials) are supported. Secrets can be referenced in connector config with ${::nested schema. service_log (boolean). Store logs for the service so that they are available in the HTTP API and console.static_ips (boolean). Use static public IP addresses.Appears on spec.userConfig.
CIDR address block, either as a string, or in a dict with an optional description field.
Required
network (string, MaxLength: 43). CIDR address block.Optional
description (string, MaxLength: 1024). Description for IP filter list entry.Appears on spec.userConfig.
Kafka Connect configuration values.
Optional
connector_client_config_override_policy (string, Enum: None, All). Defines what client configurations can be overridden by the connector. Default is None.consumer_auto_offset_reset (string, Enum: earliest, latest). What to do when there is no initial offset in Kafka or if the current offset does not exist any more on the server. Default is earliest.consumer_fetch_max_bytes (integer, Minimum: 1048576, Maximum: 104857600). Records are fetched in batches by the consumer, and if the first record batch in the first non-empty partition of the fetch is larger than this value, the record batch will still be returned to ensure that the consumer can make progress. As such, this is not a absolute maximum.consumer_isolation_level (string, Enum: read_uncommitted, read_committed). Transaction read isolation level. read_uncommitted is the default, but read_committed can be used if consume-exactly-once behavior is desired.consumer_max_partition_fetch_bytes (integer, Minimum: 1048576, Maximum: 104857600). Records are fetched in batches by the consumer.If the first record batch in the first non-empty partition of the fetch is larger than this limit, the batch will still be returned to ensure that the consumer can make progress.consumer_max_poll_interval_ms (integer, Minimum: 1, Maximum: 2147483647). The maximum delay in milliseconds between invocations of poll() when using consumer group management (defaults to 300000).consumer_max_poll_records (integer, Minimum: 1, Maximum: 10000). The maximum number of records returned in a single call to poll() (defaults to 500).offset_flush_interval_ms (integer, Minimum: 1, Maximum: 100000000). The interval at which to try committing offsets for tasks (defaults to 60000).offset_flush_timeout_ms (integer, Minimum: 1, Maximum: 2147483647). Maximum number of milliseconds to wait for records to flush and partition offset data to be committed to offset storage before cancelling the process and restoring the offset data to be committed in a future attempt (defaults to 5000).producer_batch_size (integer, Minimum: 0, Maximum: 5242880). This setting gives the upper bound of the batch size to be sent. If there are fewer than this many bytes accumulated for this partition, the producer will linger for the linger.ms time waiting for more records to show up. A batch size of zero will disable batching entirely (defaults to 16384).producer_buffer_memory (integer, Minimum: 5242880, Maximum: 134217728). The total bytes of memory the producer can use to buffer records waiting to be sent to the broker (defaults to 33554432).producer_compression_type (string, Enum: gzip, snappy, lz4, zstd, none). Specify the default compression type for producers. This configuration accepts the standard compression codecs (gzip, snappy, lz4, zstd). It additionally accepts none which is the default and equivalent to no compression.producer_linger_ms (integer, Minimum: 0, Maximum: 5000). This setting gives the upper bound on the delay for batching: once there is batch.size worth of records for a partition it will be sent immediately regardless of this setting, however if there are fewer than this many bytes accumulated for this partition the producer will linger for the specified time waiting for more records to show up. Defaults to 0.producer_max_request_size (integer, Minimum: 131072, Maximum: 67108864). This setting will limit the number of record batches the producer will send in a single request to avoid sending huge requests.scheduled_rebalance_max_delay_ms (integer, Minimum: 0, Maximum: 600000). The maximum delay that is scheduled in order to wait for the return of one or more departed workers before rebalancing and reassigning their connectors and tasks to the group. During this period the connectors and tasks of the departed workers remain unassigned. Defaults to 5 minutes.session_timeout_ms (integer, Minimum: 1, Maximum: 2147483647). The timeout in milliseconds used to detect failures when using Kafka\u2019s group management facilities (defaults to 10000).Appears on spec.userConfig.
Allow access to selected service ports from private networks.
Optional
kafka_connect (boolean). Allow clients to connect to kafka_connect with a DNS name that always resolves to the service's private IP addresses. Only available in certain network locations.prometheus (boolean). Allow clients to connect to prometheus with a DNS name that always resolves to the service's private IP addresses. Only available in certain network locations.Appears on spec.userConfig.
Allow access to selected service components through Privatelink.
Optional
jolokia (boolean). Enable jolokia.kafka_connect (boolean). Enable kafka_connect.prometheus (boolean). Enable prometheus.Appears on spec.userConfig.
Allow access to selected service ports from the public Internet.
Optional
kafka_connect (boolean). Allow clients to connect to kafka_connect from the public internet for service nodes that are in a project VPC or another type of private network.prometheus (boolean). Allow clients to connect to prometheus from the public internet for service nodes that are in a project VPC or another type of private network.Appears on spec.userConfig.
SecretProvider.
Required
name (string). Name of the secret provider. Used to reference secrets in connector config.Optional
aws (object). AWS config for Secret Provider. See below for nested schema.vault (object). Vault Config for Secret Provider. See below for nested schema.Appears on spec.userConfig.secret_providers.
AWS config for Secret Provider.
Required
auth_method (string, Enum: credentials). Auth method of the vault secret provider.region (string, MaxLength: 64). Region used to lookup secrets with AWS SecretManager.Optional
access_key (string, MaxLength: 128). Access key used to authenticate with aws.secret_key (string, MaxLength: 128). Secret key used to authenticate with aws.Appears on spec.userConfig.secret_providers.
Vault Config for Secret Provider.
Required
address (string, MinLength: 1, MaxLength: 65536). Address of the Vault server.auth_method (string, Enum: token). Auth method of the vault secret provider.Optional
engine_version (integer, Enum: 1, 2). KV Secrets Engine version of the Vault server instance.token (string, MaxLength: 256). Token used to authenticate with vault and auth method token.KafkaConnector is the Schema for the kafkaconnectors API.
Required
apiVersion (string). Value aiven.io/v1alpha1.kind (string). Value KafkaConnector.metadata (object). Data that identifies the object, including a name string and optional namespace.spec (object). KafkaConnectorSpec defines the desired state of KafkaConnector. See below for nested schema.Appears on KafkaConnector.
KafkaConnectorSpec defines the desired state of KafkaConnector.
Required
connectorClass (string, MaxLength: 1024). The Java class of the connector.project (string, Immutable, Pattern: ^[a-zA-Z0-9_-]+$, MaxLength: 63). Identifies the project this resource belongs to.serviceName (string, Immutable, Pattern: ^[a-z][-a-z0-9]+$, MaxLength: 63). Specifies the name of the service that this resource belongs to.userConfig (object, AdditionalProperties: string). The connector specific configuration To build config values from secret the template function {{ fromSecret \"name\" \"key\" }} is provided when interpreting the keys.Optional
authSecretRef (object). Authentication reference to Aiven token in a secret. See below for nested schema.Appears on spec.
Authentication reference to Aiven token in a secret.
Required
key (string, MinLength: 1).name (string, MinLength: 1).apiVersion: aiven.io/v1alpha1\nkind: KafkaSchema\nmetadata:\n name: my-schema\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n project: my-aiven-project\n serviceName: my-kafka\n subjectName: mny-subject\n compatibilityLevel: BACKWARD\n schema: |\n {\n \"doc\": \"example_doc\",\n \"fields\": [{\n \"default\": 5,\n \"doc\": \"field_doc\",\n \"name\": \"field_name\",\n \"namespace\": \"field_namespace\",\n \"type\": \"int\"\n }],\n \"name\": \"example_name\",\n \"namespace\": \"example_namespace\",\n \"type\": \"record\"\n }\nInfo
To create this resource, a Secret containing Aiven token must be created first.
Apply the resource with:
kubectl apply -f example.yaml\nVerify the newly created KafkaSchema:
kubectl get kafkaschemas my-schema\nThe output is similar to the following:
Name Service Name Project Subject Compatibility Level Version \nmy-schema my-kafka my-aiven-project mny-subject BACKWARD <version> \nKafkaSchema is the Schema for the kafkaschemas API.
Required
apiVersion (string). Value aiven.io/v1alpha1.kind (string). Value KafkaSchema.metadata (object). Data that identifies the object, including a name string and optional namespace.spec (object). KafkaSchemaSpec defines the desired state of KafkaSchema. See below for nested schema.Appears on KafkaSchema.
KafkaSchemaSpec defines the desired state of KafkaSchema.
Required
project (string, Immutable, Pattern: ^[a-zA-Z0-9_-]+$, MaxLength: 63). Identifies the project this resource belongs to.schema (string). Kafka Schema configuration should be a valid Avro Schema JSON format.serviceName (string, Immutable, Pattern: ^[a-z][-a-z0-9]+$, MaxLength: 63). Specifies the name of the service that this resource belongs to.subjectName (string, MaxLength: 63). Kafka Schema Subject name.Optional
authSecretRef (object). Authentication reference to Aiven token in a secret. See below for nested schema.compatibilityLevel (string, Enum: BACKWARD, BACKWARD_TRANSITIVE, FORWARD, FORWARD_TRANSITIVE, FULL, FULL_TRANSITIVE, NONE). Kafka Schemas compatibility level.Appears on spec.
Authentication reference to Aiven token in a secret.
Required
key (string, MinLength: 1).name (string, MinLength: 1).apiVersion: aiven.io/v1alpha1\nkind: KafkaSchemaRegistryACL\nmetadata:\n name: my-kafka-schema-registry-acl\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n project: aiven-project-name\n serviceName: my-kafka\n resource: Subject:my-topic\n username: my-user\n permission: schema_registry_read\nInfo
To create this resource, a Secret containing Aiven token must be created first.
Apply the resource with:
kubectl apply -f example.yaml\nVerify the newly created KafkaSchemaRegistryACL:
kubectl get kafkaschemaregistryacls my-kafka-schema-registry-acl\nThe output is similar to the following:
Name Project Service Name Resource Username State \nmy-kafka-schema-registry-acl aiven-project-name my-kafka Subject:my-topic my-user RUNNING \nKafkaSchemaRegistryACL is the Schema for the kafkaschemaregistryacls API.
Required
apiVersion (string). Value aiven.io/v1alpha1.kind (string). Value KafkaSchemaRegistryACL.metadata (object). Data that identifies the object, including a name string and optional namespace.spec (object). KafkaSchemaRegistryACLSpec defines the desired state of KafkaSchemaRegistryACL. See below for nested schema.Appears on KafkaSchemaRegistryACL.
KafkaSchemaRegistryACLSpec defines the desired state of KafkaSchemaRegistryACL.
Required
permission (string, Enum: schema_registry_read, schema_registry_write, Immutable).project (string, Immutable, Pattern: ^[a-zA-Z0-9_-]+$, MaxLength: 63). Identifies the project this resource belongs to.resource (string, Immutable, MaxLength: 249). Resource name pattern for the Schema Registry ACL entry.serviceName (string, Immutable, Pattern: ^[a-z][-a-z0-9]+$, MaxLength: 63). Specifies the name of the service that this resource belongs to.username (string, Immutable, MaxLength: 64). Username pattern for the ACL entry.Optional
authSecretRef (object). Authentication reference to Aiven token in a secret. See below for nested schema.Appears on spec.
Authentication reference to Aiven token in a secret.
Required
key (string, MinLength: 1).name (string, MinLength: 1).apiVersion: aiven.io/v1alpha1\nkind: KafkaTopic\nmetadata:\n name: kafka-topic\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n project: my-aiven-project\n serviceName: my-kafka\n topicName: my-kafka-topic\n\n replication: 2\n partitions: 1\n\n config:\n min_cleanable_dirty_ratio: 0.2\nInfo
To create this resource, a Secret containing Aiven token must be created first.
Apply the resource with:
kubectl apply -f example.yaml\nVerify the newly created KafkaTopic:
kubectl get kafkatopics kafka-topic\nThe output is similar to the following:
Name Service Name Project Partitions Replication \nkafka-topic my-kafka my-aiven-project 1 2 \nKafkaTopic is the Schema for the kafkatopics API.
Required
apiVersion (string). Value aiven.io/v1alpha1.kind (string). Value KafkaTopic.metadata (object). Data that identifies the object, including a name string and optional namespace.spec (object). KafkaTopicSpec defines the desired state of KafkaTopic. See below for nested schema.Appears on KafkaTopic.
KafkaTopicSpec defines the desired state of KafkaTopic.
Required
partitions (integer, Minimum: 1, Maximum: 1000000). Number of partitions to create in the topic.project (string, Immutable, Pattern: ^[a-zA-Z0-9_-]+$, MaxLength: 63). Identifies the project this resource belongs to.replication (integer, Minimum: 2). Replication factor for the topic.serviceName (string, Immutable, Pattern: ^[a-z][-a-z0-9]+$, MaxLength: 63). Specifies the name of the service that this resource belongs to.Optional
authSecretRef (object). Authentication reference to Aiven token in a secret. See below for nested schema.config (object). Kafka topic configuration. See below for nested schema.tags (array of objects). Kafka topic tags. See below for nested schema.termination_protection (boolean). It is a Kubernetes side deletion protections, which prevents the kafka topic from being deleted by Kubernetes. It is recommended to enable this for any production databases containing critical data.topicName (string, Immutable, MinLength: 1, MaxLength: 249). Topic name. If provided, is used instead of metadata.name. This field supports additional characters, has a longer length, and will replace metadata.name in future releases.Appears on spec.
Authentication reference to Aiven token in a secret.
Required
key (string, MinLength: 1).name (string, MinLength: 1).Appears on spec.
Kafka topic configuration.
Optional
cleanup_policy (string). cleanup.policy value.compression_type (string). compression.type value.delete_retention_ms (integer). delete.retention.ms value.file_delete_delay_ms (integer). file.delete.delay.ms value.flush_messages (integer). flush.messages value.flush_ms (integer). flush.ms value.index_interval_bytes (integer). index.interval.bytes value.local_retention_bytes (integer). local.retention.bytes value.local_retention_ms (integer). local.retention.ms value.max_compaction_lag_ms (integer). max.compaction.lag.ms value.max_message_bytes (integer). max.message.bytes value.message_downconversion_enable (boolean). message.downconversion.enable value.message_format_version (string). message.format.version value.message_timestamp_difference_max_ms (integer). message.timestamp.difference.max.ms value.message_timestamp_type (string). message.timestamp.type value.min_cleanable_dirty_ratio (number). min.cleanable.dirty.ratio value.min_compaction_lag_ms (integer). min.compaction.lag.ms value.min_insync_replicas (integer). min.insync.replicas value.preallocate (boolean). preallocate value.remote_storage_enable (boolean). remote_storage_enable.retention_bytes (integer). retention.bytes value.retention_ms (integer). retention.ms value.segment_bytes (integer). segment.bytes value.segment_index_bytes (integer). segment.index.bytes value.segment_jitter_ms (integer). segment.jitter.ms value.segment_ms (integer). segment.ms value.Appears on spec.
Kafka topic tags.
Required
key (string, Pattern: ^[a-zA-Z0-9_-]+$, MinLength: 1, MaxLength: 64).Optional
value (string, Pattern: ^[a-zA-Z0-9_-]+$, MaxLength: 256).apiVersion: aiven.io/v1alpha1\nkind: MySQL\nmetadata:\n name: my-mysql\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n connInfoSecretTarget:\n name: mysql-secret\n prefix: MY_SECRET_PREFIX_\n annotations:\n foo: bar\n labels:\n baz: egg\n\n project: my-aiven-project\n cloudName: google-europe-west1\n plan: business-4\n\n maintenanceWindowDow: sunday\n maintenanceWindowTime: 11:00:00\n\n userConfig:\n backup_hour: 17\n backup_minute: 11\n ip_filter:\n - network: 0.0.0.0\n description: whatever\n - network: 10.20.0.0/16\nInfo
To create this resource, a Secret containing Aiven token must be created first.
Apply the resource with:
kubectl apply -f example.yaml\nVerify the newly created MySQL:
kubectl get mysqls my-mysql\nThe output is similar to the following:
Name Project Region Plan State \nmy-mysql my-aiven-project google-europe-west1 business-4 RUNNING \nTo view the details of the Secret, use the following command:
kubectl describe secret mysql-secret\nYou can use the jq to quickly decode the Secret:
kubectl get secret mysql-secret -o json | jq '.data | map_values(@base64d)'\nThe output is similar to the following:
{\n \"MYSQL_HOST\": \"<secret>\",\n \"MYSQL_PORT\": \"<secret>\",\n \"MYSQL_DATABASE\": \"<secret>\",\n \"MYSQL_USER\": \"<secret>\",\n \"MYSQL_PASSWORD\": \"<secret>\",\n \"MYSQL_SSL_MODE\": \"<secret>\",\n \"MYSQL_URI\": \"<secret>\",\n \"MYSQL_REPLICA_URI\": \"<secret>\",\n \"MYSQL_CA_CERT\": \"<secret>\",\n}\nMySQL is the Schema for the mysqls API.
Exposes secret keys
MYSQL_HOST, MYSQL_PORT, MYSQL_DATABASE, MYSQL_USER, MYSQL_PASSWORD, MYSQL_SSL_MODE, MYSQL_URI, MYSQL_REPLICA_URI, MYSQL_CA_CERT.
Required
apiVersion (string). Value aiven.io/v1alpha1.kind (string). Value MySQL.metadata (object). Data that identifies the object, including a name string and optional namespace.spec (object). MySQLSpec defines the desired state of MySQL. See below for nested schema.Appears on MySQL.
MySQLSpec defines the desired state of MySQL.
Required
plan (string, MaxLength: 128). Subscription plan.project (string, Immutable, Pattern: ^[a-zA-Z0-9_-]+$, MaxLength: 63). Identifies the project this resource belongs to.Optional
authSecretRef (object). Authentication reference to Aiven token in a secret. See below for nested schema.cloudName (string, MaxLength: 256). Cloud the service runs in.connInfoSecretTarget (object). Secret configuration. See below for nested schema.connInfoSecretTargetDisabled (boolean, Immutable). When true, the secret containing connection information will not be created, defaults to false. This field cannot be changed after resource creation.disk_space (string, Pattern: (?i)^[1-9][0-9]*(GiB|G)?$). The disk space of the service, possible values depend on the service type, the cloud provider and the project. Reducing will result in the service re-balancing. The removal of this field does not change the value.maintenanceWindowDow (string, Enum: monday, tuesday, wednesday, thursday, friday, saturday, sunday). Day of week when maintenance operations should be performed. One monday, tuesday, wednesday, etc.maintenanceWindowTime (string, MaxLength: 8). Time of day when maintenance operations should be performed. UTC time in HH:mm:ss format.projectVPCRef (object). ProjectVPCRef reference to ProjectVPC resource to use its ID as ProjectVPCID automatically. See below for nested schema.projectVpcId (string, MaxLength: 36). Identifier of the VPC the service should be in, if any.serviceIntegrations (array of objects, Immutable, MaxItems: 1). Service integrations to specify when creating a service. Not applied after initial service creation. See below for nested schema.tags (object, AdditionalProperties: string). Tags are key-value pairs that allow you to categorize services.technicalEmails (array of objects, MaxItems: 10). Defines the email addresses that will receive alerts about upcoming maintenance updates or warnings about service instability. See below for nested schema.terminationProtection (boolean). Prevent service from being deleted. It is recommended to have this enabled for all services.userConfig (object). MySQL specific user configuration options. See below for nested schema.Appears on spec.
Authentication reference to Aiven token in a secret.
Required
key (string, MinLength: 1).name (string, MinLength: 1).Appears on spec.
Secret configuration.
Required
name (string, Immutable). Name of the secret resource to be created. By default, it is equal to the resource name.Optional
annotations (object, AdditionalProperties: string). Annotations added to the secret.labels (object, AdditionalProperties: string). Labels added to the secret.prefix (string). Prefix for the secret's keys. Added \"as is\" without any transformations. By default, is equal to the kind name in uppercase + underscore, e.g. KAFKA_, REDIS_, etc.Appears on spec.
ProjectVPCRef reference to ProjectVPC resource to use its ID as ProjectVPCID automatically.
Required
name (string, MinLength: 1).Optional
namespace (string, MinLength: 1).Appears on spec.
Service integrations to specify when creating a service. Not applied after initial service creation.
Required
integrationType (string, Enum: read_replica).sourceServiceName (string, MinLength: 1, MaxLength: 64).Appears on spec.
Defines the email addresses that will receive alerts about upcoming maintenance updates or warnings about service instability.
Required
email (string). Email address.Appears on spec.
MySQL specific user configuration options.
Optional
additional_backup_regions (array of strings, MaxItems: 1). Additional Cloud Regions for Backup Replication.admin_password (string, Immutable, Pattern: ^[a-zA-Z0-9-_]+$, MinLength: 8, MaxLength: 256). Custom password for admin user. Defaults to random string. This must be set only when a new service is being created.admin_username (string, Immutable, Pattern: ^[_A-Za-z0-9][-._A-Za-z0-9]{0,63}$, MaxLength: 64). Custom username for admin user. This must be set only when a new service is being created.backup_hour (integer, Minimum: 0, Maximum: 23). The hour of day (in UTC) when backup for the service is started. New backup is only started if previous backup has already completed.backup_minute (integer, Minimum: 0, Maximum: 59). The minute of an hour when backup for the service is started. New backup is only started if previous backup has already completed.binlog_retention_period (integer, Minimum: 600, Maximum: 86400). The minimum amount of time in seconds to keep binlog entries before deletion. This may be extended for services that require binlog entries for longer than the default for example if using the MySQL Debezium Kafka connector.ip_filter (array of objects, MaxItems: 1024). Allow incoming connections from CIDR address block, e.g. 10.20.0.0/16. See below for nested schema.migration (object). Migrate data from existing server. See below for nested schema.mysql (object). mysql.conf configuration values. See below for nested schema.mysql_version (string, Enum: 8). MySQL major version.private_access (object). Allow access to selected service ports from private networks. See below for nested schema.privatelink_access (object). Allow access to selected service components through Privatelink. See below for nested schema.project_to_fork_from (string, Immutable, Pattern: ^[a-z][-a-z0-9]{0,63}$|^$, MaxLength: 63). Name of another project to fork a service from. This has effect only when a new service is being created.public_access (object). Allow access to selected service ports from the public Internet. See below for nested schema.recovery_target_time (string, Immutable, MaxLength: 32). Recovery target time when forking a service. This has effect only when a new service is being created.service_log (boolean). Store logs for the service so that they are available in the HTTP API and console.service_to_fork_from (string, Immutable, Pattern: ^[a-z][-a-z0-9]{0,63}$|^$, MaxLength: 64). Name of another service to fork from. This has effect only when a new service is being created.static_ips (boolean). Use static public IP addresses.Appears on spec.userConfig.
CIDR address block, either as a string, or in a dict with an optional description field.
Required
network (string, MaxLength: 43). CIDR address block.Optional
description (string, MaxLength: 1024). Description for IP filter list entry.Appears on spec.userConfig.
Migrate data from existing server.
Required
host (string, MaxLength: 255). Hostname or IP address of the server where to migrate data from.port (integer, Minimum: 1, Maximum: 65535). Port number of the server where to migrate data from.Optional
dbname (string, MaxLength: 63). Database name for bootstrapping the initial connection.ignore_dbs (string, MaxLength: 2048). Comma-separated list of databases, which should be ignored during migration (supported by MySQL and PostgreSQL only at the moment).method (string, Enum: dump, replication). The migration method to be used (currently supported only by Redis, Dragonfly, MySQL and PostgreSQL service types).password (string, MaxLength: 256). Password for authentication with the server where to migrate data from.ssl (boolean). The server where to migrate data from is secured with SSL.username (string, MaxLength: 256). User name for authentication with the server where to migrate data from.Appears on spec.userConfig.
mysql.conf configuration values.
Optional
connect_timeout (integer, Minimum: 2, Maximum: 3600). The number of seconds that the mysqld server waits for a connect packet before responding with Bad handshake.default_time_zone (string, Pattern: ^([-+][\\d:]*|[\\w/]*)$, MinLength: 2, MaxLength: 100). Default server time zone as an offset from UTC (from -12:00 to +12:00), a time zone name, or SYSTEM to use the MySQL server default.group_concat_max_len (integer, Minimum: 4). The maximum permitted result length in bytes for the GROUP_CONCAT() function.information_schema_stats_expiry (integer, Minimum: 900, Maximum: 31536000). The time, in seconds, before cached statistics expire.innodb_change_buffer_max_size (integer, Minimum: 0, Maximum: 50). Maximum size for the InnoDB change buffer, as a percentage of the total size of the buffer pool. Default is 25.innodb_flush_neighbors (integer, Minimum: 0, Maximum: 2). Specifies whether flushing a page from the InnoDB buffer pool also flushes other dirty pages in the same extent (default is 1): 0 - dirty pages in the same extent are not flushed, 1 - flush contiguous dirty pages in the same extent, 2 - flush dirty pages in the same extent.innodb_ft_min_token_size (integer, Minimum: 0, Maximum: 16). Minimum length of words that are stored in an InnoDB FULLTEXT index. Changing this parameter will lead to a restart of the MySQL service.innodb_ft_server_stopword_table (string, Pattern: ^.+/.+$, MaxLength: 1024). This option is used to specify your own InnoDB FULLTEXT index stopword list for all InnoDB tables.innodb_lock_wait_timeout (integer, Minimum: 1, Maximum: 3600). The length of time in seconds an InnoDB transaction waits for a row lock before giving up. Default is 120.innodb_log_buffer_size (integer, Minimum: 1048576, Maximum: 4294967295). The size in bytes of the buffer that InnoDB uses to write to the log files on disk.innodb_online_alter_log_max_size (integer, Minimum: 65536, Maximum: 1099511627776). The upper limit in bytes on the size of the temporary log files used during online DDL operations for InnoDB tables.innodb_print_all_deadlocks (boolean). When enabled, information about all deadlocks in InnoDB user transactions is recorded in the error log. Disabled by default.innodb_read_io_threads (integer, Minimum: 1, Maximum: 64). The number of I/O threads for read operations in InnoDB. Default is 4. Changing this parameter will lead to a restart of the MySQL service.innodb_rollback_on_timeout (boolean). When enabled a transaction timeout causes InnoDB to abort and roll back the entire transaction. Changing this parameter will lead to a restart of the MySQL service.innodb_thread_concurrency (integer, Minimum: 0, Maximum: 1000). Defines the maximum number of threads permitted inside of InnoDB. Default is 0 (infinite concurrency - no limit).innodb_write_io_threads (integer, Minimum: 1, Maximum: 64). The number of I/O threads for write operations in InnoDB. Default is 4. Changing this parameter will lead to a restart of the MySQL service.interactive_timeout (integer, Minimum: 30, Maximum: 604800). The number of seconds the server waits for activity on an interactive connection before closing it.internal_tmp_mem_storage_engine (string, Enum: TempTable, MEMORY). The storage engine for in-memory internal temporary tables.long_query_time (number, Minimum: 0, Maximum: 3600). The slow_query_logs work as SQL statements that take more than long_query_time seconds to execute. Default is 10s.max_allowed_packet (integer, Minimum: 102400, Maximum: 1073741824). Size of the largest message in bytes that can be received by the server. Default is 67108864 (64M).max_heap_table_size (integer, Minimum: 1048576, Maximum: 1073741824). Limits the size of internal in-memory tables. Also set tmp_table_size. Default is 16777216 (16M).net_buffer_length (integer, Minimum: 1024, Maximum: 1048576). Start sizes of connection buffer and result buffer. Default is 16384 (16K). Changing this parameter will lead to a restart of the MySQL service.net_read_timeout (integer, Minimum: 1, Maximum: 3600). The number of seconds to wait for more data from a connection before aborting the read.net_write_timeout (integer, Minimum: 1, Maximum: 3600). The number of seconds to wait for a block to be written to a connection before aborting the write.slow_query_log (boolean). Slow query log enables capturing of slow queries. Setting slow_query_log to false also truncates the mysql.slow_log table. Default is off.sort_buffer_size (integer, Minimum: 32768, Maximum: 1073741824). Sort buffer size in bytes for ORDER BY optimization. Default is 262144 (256K).sql_mode (string, Pattern: ^[A-Z_]*(,[A-Z_]+)*$, MaxLength: 1024). Global SQL mode. Set to empty to use MySQL server defaults. When creating a new service and not setting this field Aiven default SQL mode (strict, SQL standard compliant) will be assigned.sql_require_primary_key (boolean). Require primary key to be defined for new tables or old tables modified with ALTER TABLE and fail if missing. It is recommended to always have primary keys because various functionality may break if any large table is missing them.tmp_table_size (integer, Minimum: 1048576, Maximum: 1073741824). Limits the size of internal in-memory tables. Also set max_heap_table_size. Default is 16777216 (16M).wait_timeout (integer, Minimum: 1, Maximum: 2147483). The number of seconds the server waits for activity on a noninteractive connection before closing it.Appears on spec.userConfig.
Allow access to selected service ports from private networks.
Optional
mysql (boolean). Allow clients to connect to mysql with a DNS name that always resolves to the service's private IP addresses. Only available in certain network locations.mysqlx (boolean). Allow clients to connect to mysqlx with a DNS name that always resolves to the service's private IP addresses. Only available in certain network locations.prometheus (boolean). Allow clients to connect to prometheus with a DNS name that always resolves to the service's private IP addresses. Only available in certain network locations.Appears on spec.userConfig.
Allow access to selected service components through Privatelink.
Optional
mysql (boolean). Enable mysql.mysqlx (boolean). Enable mysqlx.prometheus (boolean). Enable prometheus.Appears on spec.userConfig.
Allow access to selected service ports from the public Internet.
Optional
mysql (boolean). Allow clients to connect to mysql from the public internet for service nodes that are in a project VPC or another type of private network.mysqlx (boolean). Allow clients to connect to mysqlx from the public internet for service nodes that are in a project VPC or another type of private network.prometheus (boolean). Allow clients to connect to prometheus from the public internet for service nodes that are in a project VPC or another type of private network.apiVersion: aiven.io/v1alpha1\nkind: OpenSearch\nmetadata:\n name: my-os\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n connInfoSecretTarget:\n name: os-secret\n prefix: MY_SECRET_PREFIX_\n annotations:\n foo: bar\n labels:\n baz: egg\n\n project: my-aiven-project\n cloudName: google-europe-west1\n plan: startup-4\n disk_space: 80GiB\n\n maintenanceWindowDow: friday\n maintenanceWindowTime: 23:00:00\nInfo
To create this resource, a Secret containing Aiven token must be created first.
Apply the resource with:
kubectl apply -f example.yaml\nVerify the newly created OpenSearch:
kubectl get opensearches my-os\nThe output is similar to the following:
Name Project Region Plan State \nmy-os my-aiven-project google-europe-west1 startup-4 RUNNING \nTo view the details of the Secret, use the following command:
kubectl describe secret os-secret\nYou can use the jq to quickly decode the Secret:
kubectl get secret os-secret -o json | jq '.data | map_values(@base64d)'\nThe output is similar to the following:
{\n \"OPENSEARCH_HOST\": \"<secret>\",\n \"OPENSEARCH_PORT\": \"<secret>\",\n \"OPENSEARCH_USER\": \"<secret>\",\n \"OPENSEARCH_PASSWORD\": \"<secret>\",\n}\nOpenSearch is the Schema for the opensearches API.
Exposes secret keys
OPENSEARCH_HOST, OPENSEARCH_PORT, OPENSEARCH_USER, OPENSEARCH_PASSWORD.
Required
apiVersion (string). Value aiven.io/v1alpha1.kind (string). Value OpenSearch.metadata (object). Data that identifies the object, including a name string and optional namespace.spec (object). OpenSearchSpec defines the desired state of OpenSearch. See below for nested schema.Appears on OpenSearch.
OpenSearchSpec defines the desired state of OpenSearch.
Required
plan (string, MaxLength: 128). Subscription plan.project (string, Immutable, Pattern: ^[a-zA-Z0-9_-]+$, MaxLength: 63). Identifies the project this resource belongs to.Optional
authSecretRef (object). Authentication reference to Aiven token in a secret. See below for nested schema.cloudName (string, MaxLength: 256). Cloud the service runs in.connInfoSecretTarget (object). Secret configuration. See below for nested schema.connInfoSecretTargetDisabled (boolean, Immutable). When true, the secret containing connection information will not be created, defaults to false. This field cannot be changed after resource creation.disk_space (string, Pattern: (?i)^[1-9][0-9]*(GiB|G)?$). The disk space of the service, possible values depend on the service type, the cloud provider and the project. Reducing will result in the service re-balancing. The removal of this field does not change the value.maintenanceWindowDow (string, Enum: monday, tuesday, wednesday, thursday, friday, saturday, sunday). Day of week when maintenance operations should be performed. One monday, tuesday, wednesday, etc.maintenanceWindowTime (string, MaxLength: 8). Time of day when maintenance operations should be performed. UTC time in HH:mm:ss format.projectVPCRef (object). ProjectVPCRef reference to ProjectVPC resource to use its ID as ProjectVPCID automatically. See below for nested schema.projectVpcId (string, MaxLength: 36). Identifier of the VPC the service should be in, if any.serviceIntegrations (array of objects, Immutable, MaxItems: 1). Service integrations to specify when creating a service. Not applied after initial service creation. See below for nested schema.tags (object, AdditionalProperties: string). Tags are key-value pairs that allow you to categorize services.technicalEmails (array of objects, MaxItems: 10). Defines the email addresses that will receive alerts about upcoming maintenance updates or warnings about service instability. See below for nested schema.terminationProtection (boolean). Prevent service from being deleted. It is recommended to have this enabled for all services.userConfig (object). OpenSearch specific user configuration options. See below for nested schema.Appears on spec.
Authentication reference to Aiven token in a secret.
Required
key (string, MinLength: 1).name (string, MinLength: 1).Appears on spec.
Secret configuration.
Required
name (string, Immutable). Name of the secret resource to be created. By default, it is equal to the resource name.Optional
annotations (object, AdditionalProperties: string). Annotations added to the secret.labels (object, AdditionalProperties: string). Labels added to the secret.prefix (string). Prefix for the secret's keys. Added \"as is\" without any transformations. By default, is equal to the kind name in uppercase + underscore, e.g. KAFKA_, REDIS_, etc.Appears on spec.
ProjectVPCRef reference to ProjectVPC resource to use its ID as ProjectVPCID automatically.
Required
name (string, MinLength: 1).Optional
namespace (string, MinLength: 1).Appears on spec.
Service integrations to specify when creating a service. Not applied after initial service creation.
Required
integrationType (string, Enum: read_replica).sourceServiceName (string, MinLength: 1, MaxLength: 64).Appears on spec.
Defines the email addresses that will receive alerts about upcoming maintenance updates or warnings about service instability.
Required
email (string). Email address.Appears on spec.
OpenSearch specific user configuration options.
Optional
additional_backup_regions (array of strings, MaxItems: 1). Additional Cloud Regions for Backup Replication.custom_domain (string, MaxLength: 255). Serve the web frontend using a custom CNAME pointing to the Aiven DNS name.disable_replication_factor_adjustment (boolean). DEPRECATED: Disable automatic replication factor adjustment for multi-node services. By default, Aiven ensures all indexes are replicated at least to two nodes. Note: Due to potential data loss in case of losing a service node, this setting can no longer be activated.index_patterns (array of objects, MaxItems: 512). Index patterns. See below for nested schema.index_template (object). Template settings for all new indexes. See below for nested schema.ip_filter (array of objects, MaxItems: 1024). Allow incoming connections from CIDR address block, e.g. 10.20.0.0/16. See below for nested schema.keep_index_refresh_interval (boolean). Aiven automation resets index.refresh_interval to default value for every index to be sure that indices are always visible to search. If it doesn't fit your case, you can disable this by setting up this flag to true.max_index_count (integer, Minimum: 0). DEPRECATED: use index_patterns instead.openid (object). OpenSearch OpenID Connect Configuration. See below for nested schema.opensearch (object). OpenSearch settings. See below for nested schema.opensearch_dashboards (object). OpenSearch Dashboards settings. See below for nested schema.opensearch_version (string, Enum: 1, 2). OpenSearch major version.private_access (object). Allow access to selected service ports from private networks. See below for nested schema.privatelink_access (object). Allow access to selected service components through Privatelink. See below for nested schema.project_to_fork_from (string, Immutable, Pattern: ^[a-z][-a-z0-9]{0,63}$|^$, MaxLength: 63). Name of another project to fork a service from. This has effect only when a new service is being created.public_access (object). Allow access to selected service ports from the public Internet. See below for nested schema.recovery_basebackup_name (string, Pattern: ^[a-zA-Z0-9-_:.]+$, MaxLength: 128). Name of the basebackup to restore in forked service.saml (object). OpenSearch SAML configuration. See below for nested schema.service_log (boolean). Store logs for the service so that they are available in the HTTP API and console.service_to_fork_from (string, Immutable, Pattern: ^[a-z][-a-z0-9]{0,63}$|^$, MaxLength: 64). Name of another service to fork from. This has effect only when a new service is being created.static_ips (boolean). Use static public IP addresses.Appears on spec.userConfig.
Allows you to create glob style patterns and set a max number of indexes matching this pattern you want to keep. Creating indexes exceeding this value will cause the oldest one to get deleted. You could for example create a pattern looking like logs.? and then create index logs.1, logs.2 etc, it will delete logs.1 once you create logs.6. Do note logs.? does not apply to logs.10. Note: Setting max_index_count to 0 will do nothing and the pattern gets ignored.
Required
max_index_count (integer, Minimum: 0). Maximum number of indexes to keep.pattern (string, Pattern: ^[A-Za-z0-9-_.*?]+$, MaxLength: 1024). fnmatch pattern.Optional
sorting_algorithm (string, Enum: alphabetical, creation_date). Deletion sorting algorithm.Appears on spec.userConfig.
Template settings for all new indexes.
Optional
mapping_nested_objects_limit (integer, Minimum: 0, Maximum: 100000). The maximum number of nested JSON objects that a single document can contain across all nested types. This limit helps to prevent out of memory errors when a document contains too many nested objects. Default is 10000.number_of_replicas (integer, Minimum: 0, Maximum: 29). The number of replicas each primary shard has.number_of_shards (integer, Minimum: 1, Maximum: 1024). The number of primary shards that an index should have.Appears on spec.userConfig.
CIDR address block, either as a string, or in a dict with an optional description field.
Required
network (string, MaxLength: 43). CIDR address block.Optional
description (string, MaxLength: 1024). Description for IP filter list entry.Appears on spec.userConfig.
OpenSearch OpenID Connect Configuration.
Required
client_id (string, Pattern: ^[^\\r\\n]*$, MinLength: 1, MaxLength: 1024). The ID of the OpenID Connect client configured in your IdP. Required.client_secret (string, Pattern: ^[^\\r\\n]*$, MinLength: 1, MaxLength: 1024). The client secret of the OpenID Connect client configured in your IdP. Required.connect_url (string, MaxLength: 2048). The URL of your IdP where the Security plugin can find the OpenID Connect metadata/configuration settings.enabled (boolean). Enables or disables OpenID Connect authentication for OpenSearch. When enabled, users can authenticate using OpenID Connect with an Identity Provider.Optional
header (string, Pattern: ^[^\\r\\n]*$, MinLength: 1, MaxLength: 1024). HTTP header name of the JWT token. Optional. Default is Authorization.jwt_header (string, Pattern: ^[^\\r\\n]*$, MinLength: 1, MaxLength: 1024). The HTTP header that stores the token. Typically the Authorization header with the Bearer schema: Authorization: Bearer . Optional. Default is Authorization. jwt_url_parameter (string, Pattern: ^[^\\r\\n]*$, MinLength: 1, MaxLength: 1024). If the token is not transmitted in the HTTP header, but as an URL parameter, define the name of the parameter here. Optional.refresh_rate_limit_count (integer, Minimum: 10). The maximum number of unknown key IDs in the time frame. Default is 10. Optional.refresh_rate_limit_time_window_ms (integer, Minimum: 10000). The time frame to use when checking the maximum number of unknown key IDs, in milliseconds. Optional.Default is 10000 (10 seconds).roles_key (string, Pattern: ^[^\\r\\n]*$, MinLength: 1, MaxLength: 1024). The key in the JSON payload that stores the user\u2019s roles. The value of this key must be a comma-separated list of roles. Required only if you want to use roles in the JWT.scope (string, Pattern: ^[^\\r\\n]*$, MinLength: 1, MaxLength: 1024). The scope of the identity token issued by the IdP. Optional. Default is openid profile email address phone.subject_key (string, Pattern: ^[^\\r\\n]*$, MinLength: 1, MaxLength: 1024). The key in the JSON payload that stores the user\u2019s name. If not defined, the subject registered claim is used. Most IdP providers use the preferred_username claim. Optional.Appears on spec.userConfig.
OpenSearch settings.
Optional
action_auto_create_index_enabled (boolean). Explicitly allow or block automatic creation of indices. Defaults to true.action_destructive_requires_name (boolean). Require explicit index names when deleting.auth_failure_listeners (object). Opensearch Security Plugin Settings. See below for nested schema.cluster_max_shards_per_node (integer, Minimum: 100, Maximum: 10000). Controls the number of shards allowed in the cluster per data node.cluster_routing_allocation_node_concurrent_recoveries (integer, Minimum: 2, Maximum: 16). How many concurrent incoming/outgoing shard recoveries (normally replicas) are allowed to happen on a node. Defaults to 2.email_sender_name (string, Pattern: ^[a-zA-Z0-9-_]+$, MaxLength: 40). Sender name placeholder to be used in Opensearch Dashboards and Opensearch keystore.email_sender_password (string, Pattern: ^[^\\x00-\\x1F]+$, MaxLength: 1024). Sender password for Opensearch alerts to authenticate with SMTP server.email_sender_username (string, Pattern: ^[^\\x00-\\x1F]+$, MaxLength: 320). Sender username for Opensearch alerts.enable_security_audit (boolean). Enable/Disable security audit.http_max_content_length (integer, Minimum: 1, Maximum: 2147483647). Maximum content length for HTTP requests to the OpenSearch HTTP API, in bytes.http_max_header_size (integer, Minimum: 1024, Maximum: 262144). The max size of allowed headers, in bytes.http_max_initial_line_length (integer, Minimum: 1024, Maximum: 65536). The max length of an HTTP URL, in bytes.indices_fielddata_cache_size (integer, Minimum: 3, Maximum: 100). Relative amount. Maximum amount of heap memory used for field data cache. This is an expert setting; decreasing the value too much will increase overhead of loading field data; too much memory used for field data cache will decrease amount of heap available for other operations.indices_memory_index_buffer_size (integer, Minimum: 3, Maximum: 40). Percentage value. Default is 10%. Total amount of heap used for indexing buffer, before writing segments to disk. This is an expert setting. Too low value will slow down indexing; too high value will increase indexing performance but causes performance issues for query performance.indices_memory_max_index_buffer_size (integer, Minimum: 3, Maximum: 2048). Absolute value. Default is unbound. Doesn't work without indices.memory.index_buffer_size. Maximum amount of heap used for query cache, an absolute indices.memory.index_buffer_size maximum hard limit.indices_memory_min_index_buffer_size (integer, Minimum: 3, Maximum: 2048). Absolute value. Default is 48mb. Doesn't work without indices.memory.index_buffer_size. Minimum amount of heap used for query cache, an absolute indices.memory.index_buffer_size minimal hard limit.indices_queries_cache_size (integer, Minimum: 3, Maximum: 40). Percentage value. Default is 10%. Maximum amount of heap used for query cache. This is an expert setting. Too low value will decrease query performance and increase performance for other operations; too high value will cause issues with other OpenSearch functionality.indices_query_bool_max_clause_count (integer, Minimum: 64, Maximum: 4096). Maximum number of clauses Lucene BooleanQuery can have. The default value (1024) is relatively high, and increasing it may cause performance issues. Investigate other approaches first before increasing this value.indices_recovery_max_bytes_per_sec (integer, Minimum: 40, Maximum: 400). Limits total inbound and outbound recovery traffic for each node. Applies to both peer recoveries as well as snapshot recoveries (i.e., restores from a snapshot). Defaults to 40mb.indices_recovery_max_concurrent_file_chunks (integer, Minimum: 2, Maximum: 5). Number of file chunks sent in parallel for each recovery. Defaults to 2.ism_enabled (boolean). Specifies whether ISM is enabled or not.ism_history_enabled (boolean). Specifies whether audit history is enabled or not. The logs from ISM are automatically indexed to a logs document.ism_history_max_age (integer, Minimum: 1, Maximum: 2147483647). The maximum age before rolling over the audit history index in hours.ism_history_max_docs (integer, Minimum: 1). The maximum number of documents before rolling over the audit history index.ism_history_rollover_check_period (integer, Minimum: 1, Maximum: 2147483647). The time between rollover checks for the audit history index in hours.ism_history_rollover_retention_period (integer, Minimum: 1, Maximum: 2147483647). How long audit history indices are kept in days.knn_memory_circuit_breaker_enabled (boolean). Enable or disable KNN memory circuit breaker. Defaults to true.knn_memory_circuit_breaker_limit (integer, Minimum: 3, Maximum: 100). Maximum amount of memory that can be used for KNN index. Defaults to 50% of the JVM heap size.override_main_response_version (boolean). Compatibility mode sets OpenSearch to report its version as 7.10 so clients continue to work. Default is false.plugins_alerting_filter_by_backend_roles (boolean). Enable or disable filtering of alerting by backend roles. Requires Security plugin. Defaults to false.reindex_remote_whitelist (array of strings, MaxItems: 32). Whitelisted addresses for reindexing. Changing this value will cause all OpenSearch instances to restart.script_max_compilations_rate (string, MaxLength: 1024). Script compilation circuit breaker limits the number of inline script compilations within a period of time. Default is use-context.search_max_buckets (integer, Minimum: 1, Maximum: 1000000). Maximum number of aggregation buckets allowed in a single response. OpenSearch default value is used when this is not defined.thread_pool_analyze_queue_size (integer, Minimum: 10, Maximum: 2000). Size for the thread pool queue. See documentation for exact details.thread_pool_analyze_size (integer, Minimum: 1, Maximum: 128). Size for the thread pool. See documentation for exact details. Do note this may have maximum value depending on CPU count - value is automatically lowered if set to higher than maximum value.thread_pool_force_merge_size (integer, Minimum: 1, Maximum: 128). Size for the thread pool. See documentation for exact details. Do note this may have maximum value depending on CPU count - value is automatically lowered if set to higher than maximum value.thread_pool_get_queue_size (integer, Minimum: 10, Maximum: 2000). Size for the thread pool queue. See documentation for exact details.thread_pool_get_size (integer, Minimum: 1, Maximum: 128). Size for the thread pool. See documentation for exact details. Do note this may have maximum value depending on CPU count - value is automatically lowered if set to higher than maximum value.thread_pool_search_queue_size (integer, Minimum: 10, Maximum: 2000). Size for the thread pool queue. See documentation for exact details.thread_pool_search_size (integer, Minimum: 1, Maximum: 128). Size for the thread pool. See documentation for exact details. Do note this may have maximum value depending on CPU count - value is automatically lowered if set to higher than maximum value.thread_pool_search_throttled_queue_size (integer, Minimum: 10, Maximum: 2000). Size for the thread pool queue. See documentation for exact details.thread_pool_search_throttled_size (integer, Minimum: 1, Maximum: 128). Size for the thread pool. See documentation for exact details. Do note this may have maximum value depending on CPU count - value is automatically lowered if set to higher than maximum value.thread_pool_write_queue_size (integer, Minimum: 10, Maximum: 2000). Size for the thread pool queue. See documentation for exact details.thread_pool_write_size (integer, Minimum: 1, Maximum: 128). Size for the thread pool. See documentation for exact details. Do note this may have maximum value depending on CPU count - value is automatically lowered if set to higher than maximum value.Appears on spec.userConfig.opensearch.
Opensearch Security Plugin Settings.
Optional
internal_authentication_backend_limiting (object). See below for nested schema.ip_rate_limiting (object). IP address rate limiting settings. See below for nested schema.Appears on spec.userConfig.opensearch.auth_failure_listeners.
Optional
allowed_tries (integer, Minimum: 0, Maximum: 2147483647). The number of login attempts allowed before login is blocked.authentication_backend (string, Enum: internal, MaxLength: 1024). internal_authentication_backend_limiting.authentication_backend.block_expiry_seconds (integer, Minimum: 0, Maximum: 2147483647). The duration of time that login remains blocked after a failed login.max_blocked_clients (integer, Minimum: 0, Maximum: 2147483647). internal_authentication_backend_limiting.max_blocked_clients.max_tracked_clients (integer, Minimum: 0, Maximum: 2147483647). The maximum number of tracked IP addresses that have failed login.time_window_seconds (integer, Minimum: 0, Maximum: 2147483647). The window of time in which the value for allowed_tries is enforced.type (string, Enum: username, MaxLength: 1024). internal_authentication_backend_limiting.type.Appears on spec.userConfig.opensearch.auth_failure_listeners.
IP address rate limiting settings.
Optional
allowed_tries (integer, Minimum: 1, Maximum: 2147483647). The number of login attempts allowed before login is blocked.block_expiry_seconds (integer, Minimum: 1, Maximum: 36000). The duration of time that login remains blocked after a failed login.max_blocked_clients (integer, Minimum: 0, Maximum: 2147483647). The maximum number of blocked IP addresses.max_tracked_clients (integer, Minimum: 0, Maximum: 2147483647). The maximum number of tracked IP addresses that have failed login.time_window_seconds (integer, Minimum: 1, Maximum: 36000). The window of time in which the value for allowed_tries is enforced.type (string, Enum: ip, MaxLength: 1024). The type of rate limiting.Appears on spec.userConfig.
OpenSearch Dashboards settings.
Optional
enabled (boolean). Enable or disable OpenSearch Dashboards.max_old_space_size (integer, Minimum: 64, Maximum: 2048). Limits the maximum amount of memory (in MiB) the OpenSearch Dashboards process can use. This sets the max_old_space_size option of the nodejs running the OpenSearch Dashboards. Note: the memory reserved by OpenSearch Dashboards is not available for OpenSearch.opensearch_request_timeout (integer, Minimum: 5000, Maximum: 120000). Timeout in milliseconds for requests made by OpenSearch Dashboards towards OpenSearch.Appears on spec.userConfig.
Allow access to selected service ports from private networks.
Optional
opensearch (boolean). Allow clients to connect to opensearch with a DNS name that always resolves to the service's private IP addresses. Only available in certain network locations.opensearch_dashboards (boolean). Allow clients to connect to opensearch_dashboards with a DNS name that always resolves to the service's private IP addresses. Only available in certain network locations.prometheus (boolean). Allow clients to connect to prometheus with a DNS name that always resolves to the service's private IP addresses. Only available in certain network locations.Appears on spec.userConfig.
Allow access to selected service components through Privatelink.
Optional
opensearch (boolean). Enable opensearch.opensearch_dashboards (boolean). Enable opensearch_dashboards.prometheus (boolean). Enable prometheus.Appears on spec.userConfig.
Allow access to selected service ports from the public Internet.
Optional
opensearch (boolean). Allow clients to connect to opensearch from the public internet for service nodes that are in a project VPC or another type of private network.opensearch_dashboards (boolean). Allow clients to connect to opensearch_dashboards from the public internet for service nodes that are in a project VPC or another type of private network.prometheus (boolean). Allow clients to connect to prometheus from the public internet for service nodes that are in a project VPC or another type of private network.Appears on spec.userConfig.
OpenSearch SAML configuration.
Required
enabled (boolean). Enables or disables SAML-based authentication for OpenSearch. When enabled, users can authenticate using SAML with an Identity Provider.idp_entity_id (string, Pattern: ^[^\\r\\n]*$, MinLength: 1, MaxLength: 1024). The unique identifier for the Identity Provider (IdP) entity that is used for SAML authentication. This value is typically provided by the IdP.idp_metadata_url (string, MinLength: 1, MaxLength: 2048). The URL of the SAML metadata for the Identity Provider (IdP). This is used to configure SAML-based authentication with the IdP.sp_entity_id (string, Pattern: ^[^\\r\\n]*$, MinLength: 1, MaxLength: 1024). The unique identifier for the Service Provider (SP) entity that is used for SAML authentication. This value is typically provided by the SP.Optional
idp_pemtrustedcas_content (string, MaxLength: 16384). This parameter specifies the PEM-encoded root certificate authority (CA) content for the SAML identity provider (IdP) server verification. The root CA content is used to verify the SSL/TLS certificate presented by the server.roles_key (string, Pattern: ^[^\\r\\n]*$, MinLength: 1, MaxLength: 256). Optional. Specifies the attribute in the SAML response where role information is stored, if available. Role attributes are not required for SAML authentication, but can be included in SAML assertions by most Identity Providers (IdPs) to determine user access levels or permissions.subject_key (string, Pattern: ^[^\\r\\n]*$, MinLength: 1, MaxLength: 256). Optional. Specifies the attribute in the SAML response where the subject identifier is stored. If not configured, the NameID attribute is used by default.apiVersion: aiven.io/v1alpha1\nkind: PostgreSQL\nmetadata:\n name: my-postgresql\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n connInfoSecretTarget:\n name: postgresql-secret\n prefix: MY_SECRET_PREFIX_\n annotations:\n foo: bar\n labels:\n baz: egg\n\n project: aiven-project-name\n cloudName: google-europe-west1\n plan: startup-4\n\n maintenanceWindowDow: sunday\n maintenanceWindowTime: 11:00:00\n\n userConfig:\n pg_version: \"15\"\nInfo
To create this resource, a Secret containing Aiven token must be created first.
Apply the resource with:
kubectl apply -f example.yaml\nVerify the newly created PostgreSQL:
kubectl get postgresqls my-postgresql\nThe output is similar to the following:
Name Project Region Plan State \nmy-postgresql aiven-project-name google-europe-west1 startup-4 RUNNING \nTo view the details of the Secret, use the following command:
kubectl describe secret postgresql-secret\nYou can use the jq to quickly decode the Secret:
kubectl get secret postgresql-secret -o json | jq '.data | map_values(@base64d)'\nThe output is similar to the following:
{\n \"POSTGRESQL_HOST\": \"<secret>\",\n \"POSTGRESQL_PORT\": \"<secret>\",\n \"POSTGRESQL_DATABASE\": \"<secret>\",\n \"POSTGRESQL_USER\": \"<secret>\",\n \"POSTGRESQL_PASSWORD\": \"<secret>\",\n \"POSTGRESQL_SSLMODE\": \"<secret>\",\n \"POSTGRESQL_DATABASE_URI\": \"<secret>\",\n \"POSTGRESQL_CA_CERT\": \"<secret>\",\n}\nPostgreSQL is the Schema for the postgresql API.
Exposes secret keys
POSTGRESQL_HOST, POSTGRESQL_PORT, POSTGRESQL_DATABASE, POSTGRESQL_USER, POSTGRESQL_PASSWORD, POSTGRESQL_SSLMODE, POSTGRESQL_DATABASE_URI, POSTGRESQL_CA_CERT.
Required
apiVersion (string). Value aiven.io/v1alpha1.kind (string). Value PostgreSQL.metadata (object). Data that identifies the object, including a name string and optional namespace.spec (object). PostgreSQLSpec defines the desired state of postgres instance. See below for nested schema.Appears on PostgreSQL.
PostgreSQLSpec defines the desired state of postgres instance.
Required
plan (string, MaxLength: 128). Subscription plan.project (string, Immutable, Pattern: ^[a-zA-Z0-9_-]+$, MaxLength: 63). Identifies the project this resource belongs to.Optional
authSecretRef (object). Authentication reference to Aiven token in a secret. See below for nested schema.cloudName (string, MaxLength: 256). Cloud the service runs in.connInfoSecretTarget (object). Secret configuration. See below for nested schema.connInfoSecretTargetDisabled (boolean, Immutable). When true, the secret containing connection information will not be created, defaults to false. This field cannot be changed after resource creation.disk_space (string, Pattern: (?i)^[1-9][0-9]*(GiB|G)?$). The disk space of the service, possible values depend on the service type, the cloud provider and the project. Reducing will result in the service re-balancing. The removal of this field does not change the value.maintenanceWindowDow (string, Enum: monday, tuesday, wednesday, thursday, friday, saturday, sunday). Day of week when maintenance operations should be performed. One monday, tuesday, wednesday, etc.maintenanceWindowTime (string, MaxLength: 8). Time of day when maintenance operations should be performed. UTC time in HH:mm:ss format.projectVPCRef (object). ProjectVPCRef reference to ProjectVPC resource to use its ID as ProjectVPCID automatically. See below for nested schema.projectVpcId (string, MaxLength: 36). Identifier of the VPC the service should be in, if any.serviceIntegrations (array of objects, Immutable, MaxItems: 1). Service integrations to specify when creating a service. Not applied after initial service creation. See below for nested schema.tags (object, AdditionalProperties: string). Tags are key-value pairs that allow you to categorize services.technicalEmails (array of objects, MaxItems: 10). Defines the email addresses that will receive alerts about upcoming maintenance updates or warnings about service instability. See below for nested schema.terminationProtection (boolean). Prevent service from being deleted. It is recommended to have this enabled for all services.userConfig (object). PostgreSQL specific user configuration options. See below for nested schema.Appears on spec.
Authentication reference to Aiven token in a secret.
Required
key (string, MinLength: 1).name (string, MinLength: 1).Appears on spec.
Secret configuration.
Required
name (string, Immutable). Name of the secret resource to be created. By default, it is equal to the resource name.Optional
annotations (object, AdditionalProperties: string). Annotations added to the secret.labels (object, AdditionalProperties: string). Labels added to the secret.prefix (string). Prefix for the secret's keys. Added \"as is\" without any transformations. By default, is equal to the kind name in uppercase + underscore, e.g. KAFKA_, REDIS_, etc.Appears on spec.
ProjectVPCRef reference to ProjectVPC resource to use its ID as ProjectVPCID automatically.
Required
name (string, MinLength: 1).Optional
namespace (string, MinLength: 1).Appears on spec.
Service integrations to specify when creating a service. Not applied after initial service creation.
Required
integrationType (string, Enum: read_replica).sourceServiceName (string, MinLength: 1, MaxLength: 64).Appears on spec.
Defines the email addresses that will receive alerts about upcoming maintenance updates or warnings about service instability.
Required
email (string). Email address.Appears on spec.
PostgreSQL specific user configuration options.
Optional
additional_backup_regions (array of strings, MaxItems: 1). Additional Cloud Regions for Backup Replication.admin_password (string, Immutable, Pattern: ^[a-zA-Z0-9-_]+$, MinLength: 8, MaxLength: 256). Custom password for admin user. Defaults to random string. This must be set only when a new service is being created.admin_username (string, Immutable, Pattern: ^[_A-Za-z0-9][-._A-Za-z0-9]{0,63}$, MaxLength: 64). Custom username for admin user. This must be set only when a new service is being created.backup_hour (integer, Minimum: 0, Maximum: 23). The hour of day (in UTC) when backup for the service is started. New backup is only started if previous backup has already completed.backup_minute (integer, Minimum: 0, Maximum: 59). The minute of an hour when backup for the service is started. New backup is only started if previous backup has already completed.enable_ipv6 (boolean). Register AAAA DNS records for the service, and allow IPv6 packets to service ports.ip_filter (array of objects, MaxItems: 1024). Allow incoming connections from CIDR address block, e.g. 10.20.0.0/16. See below for nested schema.migration (object). Migrate data from existing server. See below for nested schema.pg (object). postgresql.conf configuration values. See below for nested schema.pg_qualstats (object). Deprecated. System-wide settings for the pg_qualstats extension. See below for nested schema.pg_read_replica (boolean). Should the service which is being forked be a read replica (deprecated, use read_replica service integration instead).pg_service_to_fork_from (string, Immutable, Pattern: ^[a-z][-a-z0-9]{0,63}$|^$, MaxLength: 64). Name of the PG Service from which to fork (deprecated, use service_to_fork_from). This has effect only when a new service is being created.pg_stat_monitor_enable (boolean). Enable the pg_stat_monitor extension. Enabling this extension will cause the cluster to be restarted.When this extension is enabled, pg_stat_statements results for utility commands are unreliable.pg_version (string, Enum: 12, 13, 14, 15, 16). PostgreSQL major version.pgaudit (object). Deprecated. System-wide settings for the pgaudit extension. See below for nested schema.pgbouncer (object). PGBouncer connection pooling settings. See below for nested schema.pglookout (object). System-wide settings for pglookout. See below for nested schema.private_access (object). Allow access to selected service ports from private networks. See below for nested schema.privatelink_access (object). Allow access to selected service components through Privatelink. See below for nested schema.project_to_fork_from (string, Immutable, Pattern: ^[a-z][-a-z0-9]{0,63}$|^$, MaxLength: 63). Name of another project to fork a service from. This has effect only when a new service is being created.public_access (object). Allow access to selected service ports from the public Internet. See below for nested schema.recovery_target_time (string, Immutable, MaxLength: 32). Recovery target time when forking a service. This has effect only when a new service is being created.service_log (boolean). Store logs for the service so that they are available in the HTTP API and console.service_to_fork_from (string, Immutable, Pattern: ^[a-z][-a-z0-9]{0,63}$|^$, MaxLength: 64). Name of another service to fork from. This has effect only when a new service is being created.shared_buffers_percentage (number, Minimum: 20, Maximum: 60). Percentage of total RAM that the database server uses for shared memory buffers. Valid range is 20-60 (float), which corresponds to 20% - 60%. This setting adjusts the shared_buffers configuration value.static_ips (boolean). Use static public IP addresses.synchronous_replication (string, Enum: quorum, off). Synchronous replication type. Note that the service plan also needs to support synchronous replication.timescaledb (object). System-wide settings for the timescaledb extension. See below for nested schema.variant (string, Enum: aiven, timescale). Variant of the PostgreSQL service, may affect the features that are exposed by default.work_mem (integer, Minimum: 1, Maximum: 1024). Sets the maximum amount of memory to be used by a query operation (such as a sort or hash table) before writing to temporary disk files, in MB. Default is 1MB + 0.075% of total RAM (up to 32MB).Appears on spec.userConfig.
CIDR address block, either as a string, or in a dict with an optional description field.
Required
network (string, MaxLength: 43). CIDR address block.Optional
description (string, MaxLength: 1024). Description for IP filter list entry.Appears on spec.userConfig.
Migrate data from existing server.
Required
host (string, MaxLength: 255). Hostname or IP address of the server where to migrate data from.port (integer, Minimum: 1, Maximum: 65535). Port number of the server where to migrate data from.Optional
dbname (string, MaxLength: 63). Database name for bootstrapping the initial connection.ignore_dbs (string, MaxLength: 2048). Comma-separated list of databases, which should be ignored during migration (supported by MySQL and PostgreSQL only at the moment).method (string, Enum: dump, replication). The migration method to be used (currently supported only by Redis, Dragonfly, MySQL and PostgreSQL service types).password (string, MaxLength: 256). Password for authentication with the server where to migrate data from.ssl (boolean). The server where to migrate data from is secured with SSL.username (string, MaxLength: 256). User name for authentication with the server where to migrate data from.Appears on spec.userConfig.
postgresql.conf configuration values.
Optional
autovacuum_analyze_scale_factor (number, Minimum: 0, Maximum: 1). Specifies a fraction of the table size to add to autovacuum_analyze_threshold when deciding whether to trigger an ANALYZE. The default is 0.2 (20% of table size).autovacuum_analyze_threshold (integer, Minimum: 0, Maximum: 2147483647). Specifies the minimum number of inserted, updated or deleted tuples needed to trigger an ANALYZE in any one table. The default is 50 tuples.autovacuum_freeze_max_age (integer, Minimum: 200000000, Maximum: 1500000000). Specifies the maximum age (in transactions) that a table's pg_class.relfrozenxid field can attain before a VACUUM operation is forced to prevent transaction ID wraparound within the table. Note that the system will launch autovacuum processes to prevent wraparound even when autovacuum is otherwise disabled. This parameter will cause the server to be restarted.autovacuum_max_workers (integer, Minimum: 1, Maximum: 20). Specifies the maximum number of autovacuum processes (other than the autovacuum launcher) that may be running at any one time. The default is three. This parameter can only be set at server start.autovacuum_naptime (integer, Minimum: 1, Maximum: 86400). Specifies the minimum delay between autovacuum runs on any given database. The delay is measured in seconds, and the default is one minute.autovacuum_vacuum_cost_delay (integer, Minimum: -1, Maximum: 100). Specifies the cost delay value that will be used in automatic VACUUM operations. If -1 is specified, the regular vacuum_cost_delay value will be used. The default value is 20 milliseconds.autovacuum_vacuum_cost_limit (integer, Minimum: -1, Maximum: 10000). Specifies the cost limit value that will be used in automatic VACUUM operations. If -1 is specified (which is the default), the regular vacuum_cost_limit value will be used.autovacuum_vacuum_scale_factor (number, Minimum: 0, Maximum: 1). Specifies a fraction of the table size to add to autovacuum_vacuum_threshold when deciding whether to trigger a VACUUM. The default is 0.2 (20% of table size).autovacuum_vacuum_threshold (integer, Minimum: 0, Maximum: 2147483647). Specifies the minimum number of updated or deleted tuples needed to trigger a VACUUM in any one table. The default is 50 tuples.bgwriter_delay (integer, Minimum: 10, Maximum: 10000). Specifies the delay between activity rounds for the background writer in milliseconds. Default is 200.bgwriter_flush_after (integer, Minimum: 0, Maximum: 2048). Whenever more than bgwriter_flush_after bytes have been written by the background writer, attempt to force the OS to issue these writes to the underlying storage. Specified in kilobytes, default is 512. Setting of 0 disables forced writeback.bgwriter_lru_maxpages (integer, Minimum: 0, Maximum: 1073741823). In each round, no more than this many buffers will be written by the background writer. Setting this to zero disables background writing. Default is 100.bgwriter_lru_multiplier (number, Minimum: 0, Maximum: 10). The average recent need for new buffers is multiplied by bgwriter_lru_multiplier to arrive at an estimate of the number that will be needed during the next round, (up to bgwriter_lru_maxpages). 1.0 represents a \u201cjust in time\u201d policy of writing exactly the number of buffers predicted to be needed. Larger values provide some cushion against spikes in demand, while smaller values intentionally leave writes to be done by server processes. The default is 2.0.deadlock_timeout (integer, Minimum: 500, Maximum: 1800000). This is the amount of time, in milliseconds, to wait on a lock before checking to see if there is a deadlock condition.default_toast_compression (string, Enum: lz4, pglz). Specifies the default TOAST compression method for values of compressible columns (the default is lz4).idle_in_transaction_session_timeout (integer, Minimum: 0, Maximum: 604800000). Time out sessions with open transactions after this number of milliseconds.jit (boolean). Controls system-wide use of Just-in-Time Compilation (JIT).log_autovacuum_min_duration (integer, Minimum: -1, Maximum: 2147483647). Causes each action executed by autovacuum to be logged if it ran for at least the specified number of milliseconds. Setting this to zero logs all autovacuum actions. Minus-one (the default) disables logging autovacuum actions.log_error_verbosity (string, Enum: TERSE, DEFAULT, VERBOSE). Controls the amount of detail written in the server log for each message that is logged.log_line_prefix (string, Enum: 'pid=%p,user=%u,db=%d,app=%a,client=%h ', '%t [%p]: [%l-1] user=%u,db=%d,app=%a,client=%h ', '%m [%p] %q[user=%u,db=%d,app=%a] ', 'pid=%p,user=%u,db=%d,app=%a,client=%h,txid=%x,qid=%Q '). Choose from one of the available log formats.log_min_duration_statement (integer, Minimum: -1, Maximum: 86400000). Log statements that take more than this number of milliseconds to run, -1 disables.log_temp_files (integer, Minimum: -1, Maximum: 2147483647). Log statements for each temporary file created larger than this number of kilobytes, -1 disables.max_files_per_process (integer, Minimum: 1000, Maximum: 4096). PostgreSQL maximum number of files that can be open per process.max_locks_per_transaction (integer, Minimum: 64, Maximum: 6400). PostgreSQL maximum locks per transaction.max_logical_replication_workers (integer, Minimum: 4, Maximum: 64). PostgreSQL maximum logical replication workers (taken from the pool of max_parallel_workers).max_parallel_workers (integer, Minimum: 0, Maximum: 96). Sets the maximum number of workers that the system can support for parallel queries.max_parallel_workers_per_gather (integer, Minimum: 0, Maximum: 96). Sets the maximum number of workers that can be started by a single Gather or Gather Merge node.max_pred_locks_per_transaction (integer, Minimum: 64, Maximum: 5120). PostgreSQL maximum predicate locks per transaction.max_prepared_transactions (integer, Minimum: 0, Maximum: 10000). PostgreSQL maximum prepared transactions.max_replication_slots (integer, Minimum: 8, Maximum: 64). PostgreSQL maximum replication slots.max_slot_wal_keep_size (integer, Minimum: -1, Maximum: 2147483647). PostgreSQL maximum WAL size (MB) reserved for replication slots. Default is -1 (unlimited). wal_keep_size minimum WAL size setting takes precedence over this.max_stack_depth (integer, Minimum: 2097152, Maximum: 6291456). Maximum depth of the stack in bytes.max_standby_archive_delay (integer, Minimum: 1, Maximum: 43200000). Max standby archive delay in milliseconds.max_standby_streaming_delay (integer, Minimum: 1, Maximum: 43200000). Max standby streaming delay in milliseconds.max_wal_senders (integer, Minimum: 20, Maximum: 64). PostgreSQL maximum WAL senders.max_worker_processes (integer, Minimum: 8, Maximum: 96). Sets the maximum number of background processes that the system can support.pg_partman_bgw.interval (integer, Minimum: 3600, Maximum: 604800). Sets the time interval to run pg_partman's scheduled tasks.pg_partman_bgw.role (string, Pattern: ^[_A-Za-z0-9][-._A-Za-z0-9]{0,63}$, MaxLength: 64). Controls which role to use for pg_partman's scheduled background tasks.pg_stat_monitor.pgsm_enable_query_plan (boolean). Enables or disables query plan monitoring.pg_stat_monitor.pgsm_max_buckets (integer, Minimum: 1, Maximum: 10). Sets the maximum number of buckets.pg_stat_statements.track (string, Enum: all, top, none). Controls which statements are counted. Specify top to track top-level statements (those issued directly by clients), all to also track nested statements (such as statements invoked within functions), or none to disable statement statistics collection. The default value is top.temp_file_limit (integer, Minimum: -1, Maximum: 2147483647). PostgreSQL temporary file limit in KiB, -1 for unlimited.timezone (string, Pattern: ^[\\w/]*$, MaxLength: 64). PostgreSQL service timezone.track_activity_query_size (integer, Minimum: 1024, Maximum: 10240). Specifies the number of bytes reserved to track the currently executing command for each active session.track_commit_timestamp (string, Enum: off, on). Record commit time of transactions.track_functions (string, Enum: all, pl, none). Enables tracking of function call counts and time used.track_io_timing (string, Enum: off, on). Enables timing of database I/O calls. This parameter is off by default, because it will repeatedly query the operating system for the current time, which may cause significant overhead on some platforms.wal_sender_timeout (integer). Terminate replication connections that are inactive for longer than this amount of time, in milliseconds. Setting this value to zero disables the timeout.wal_writer_delay (integer, Minimum: 10, Maximum: 200). WAL flush interval in milliseconds. Note that setting this value to lower than the default 200ms may negatively impact performance.Appears on spec.userConfig.
Deprecated. System-wide settings for the pg_qualstats extension.
Optional
enabled (boolean). Deprecated. Enable / Disable pg_qualstats.min_err_estimate_num (integer, Minimum: 0). Deprecated. Error estimation num threshold to save quals.min_err_estimate_ratio (integer, Minimum: 0). Deprecated. Error estimation ratio threshold to save quals.track_constants (boolean). Deprecated. Enable / Disable pg_qualstats constants tracking.track_pg_catalog (boolean). Deprecated. Track quals on system catalogs too.Appears on spec.userConfig.
Deprecated. System-wide settings for the pgaudit extension.
Optional
feature_enabled (boolean). Deprecated. Enable pgaudit extension. When enabled, pgaudit extension will be automatically installed.Otherwise, extension will be uninstalled but auditing configurations will be preserved.log (array of strings). Deprecated. Specifies which classes of statements will be logged by session audit logging.log_catalog (boolean). Deprecated. Specifies that session logging should be enabled in the casewhere all relations in a statement are in pg_catalog.log_client (boolean). Deprecated. Specifies whether log messages will be visible to a client process such as psql.log_level (string). Deprecated. Specifies the log level that will be used for log entries.log_max_string_length (integer, Minimum: -1, Maximum: 102400). Deprecated. Crop parameters representation and whole statements if they exceed this threshold. A (default) value of -1 disable the truncation.log_nested_statements (boolean). Deprecated. This GUC allows to turn off logging nested statements, that is, statements that are executed as part of another ExecutorRun.log_parameter (boolean). Deprecated. Specifies that audit logging should include the parameters that were passed with the statement.log_parameter_max_size (integer). Deprecated. Specifies that parameter values longer than this setting (in bytes) should not be logged, but replaced with . log_relation (boolean). Deprecated. Specifies whether session audit logging should create a separate log entry for each relation (TABLE, VIEW, etc.) referenced in a SELECT or DML statement.log_rows (boolean). Deprecated. Specifies that audit logging should include the rows retrieved or affected by a statement. When enabled the rows field will be included after the parameter field.log_statement (boolean). Deprecated. Specifies whether logging will include the statement text and parameters (if enabled).log_statement_once (boolean). Deprecated. Specifies whether logging will include the statement text and parameters with the first log entry for a statement/substatement combination or with every entry.role (string, Pattern: ^[_A-Za-z0-9][-._A-Za-z0-9]{0,63}$, MaxLength: 64). Deprecated. Specifies the master role to use for object audit logging.Appears on spec.userConfig.
PGBouncer connection pooling settings.
Optional
autodb_idle_timeout (integer, Minimum: 0, Maximum: 86400). If the automatically created database pools have been unused this many seconds, they are freed. If 0 then timeout is disabled. [seconds].autodb_max_db_connections (integer, Minimum: 0, Maximum: 2147483647). Do not allow more than this many server connections per database (regardless of user). Setting it to 0 means unlimited.autodb_pool_mode (string, Enum: session, transaction, statement). PGBouncer pool mode.autodb_pool_size (integer, Minimum: 0, Maximum: 10000). If non-zero then create automatically a pool of that size per user when a pool doesn't exist.ignore_startup_parameters (array of strings, MaxItems: 32). List of parameters to ignore when given in startup packet.min_pool_size (integer, Minimum: 0, Maximum: 10000). Add more server connections to pool if below this number. Improves behavior when usual load comes suddenly back after period of total inactivity. The value is effectively capped at the pool size.server_idle_timeout (integer, Minimum: 0, Maximum: 86400). If a server connection has been idle more than this many seconds it will be dropped. If 0 then timeout is disabled. [seconds].server_lifetime (integer, Minimum: 60, Maximum: 86400). The pooler will close an unused server connection that has been connected longer than this. [seconds].server_reset_query_always (boolean). Run server_reset_query (DISCARD ALL) in all pooling modes.Appears on spec.userConfig.
System-wide settings for pglookout.
Required
max_failover_replication_time_lag (integer, Minimum: 10). Number of seconds of master unavailability before triggering database failover to standby.Appears on spec.userConfig.
Allow access to selected service ports from private networks.
Optional
pg (boolean). Allow clients to connect to pg with a DNS name that always resolves to the service's private IP addresses. Only available in certain network locations.pgbouncer (boolean). Allow clients to connect to pgbouncer with a DNS name that always resolves to the service's private IP addresses. Only available in certain network locations.prometheus (boolean). Allow clients to connect to prometheus with a DNS name that always resolves to the service's private IP addresses. Only available in certain network locations.Appears on spec.userConfig.
Allow access to selected service components through Privatelink.
Optional
pg (boolean). Enable pg.pgbouncer (boolean). Enable pgbouncer.prometheus (boolean). Enable prometheus.Appears on spec.userConfig.
Allow access to selected service ports from the public Internet.
Optional
pg (boolean). Allow clients to connect to pg from the public internet for service nodes that are in a project VPC or another type of private network.pgbouncer (boolean). Allow clients to connect to pgbouncer from the public internet for service nodes that are in a project VPC or another type of private network.prometheus (boolean). Allow clients to connect to prometheus from the public internet for service nodes that are in a project VPC or another type of private network.Appears on spec.userConfig.
System-wide settings for the timescaledb extension.
Required
max_background_workers (integer, Minimum: 1, Maximum: 4096). The number of background workers for timescaledb operations. You should configure this setting to the sum of your number of databases and the total number of concurrent background workers you want running at any given point in time.apiVersion: aiven.io/v1alpha1\nkind: Project\nmetadata:\n name: my-project\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n connInfoSecretTarget:\n name: project-secret\n prefix: MY_SECRET_PREFIX_\n annotations:\n foo: bar\n labels:\n baz: egg\n\n tags:\n env: prod\n\n billingAddress: NYC\n cloud: aws-eu-west-1\nInfo
To create this resource, a Secret containing Aiven token must be created first.
Apply the resource with:
kubectl apply -f example.yaml\nVerify the newly created Project:
kubectl get projects my-project\nThe output is similar to the following:
Name \nmy-project \nTo view the details of the Secret, use the following command:
kubectl describe secret project-secret\nYou can use the jq to quickly decode the Secret:
kubectl get secret project-secret -o json | jq '.data | map_values(@base64d)'\nThe output is similar to the following:
{\n \"PROJECT_CA_CERT\": \"<secret>\",\n}\nProject is the Schema for the projects API.
Exposes secret keys
PROJECT_CA_CERT.
Required
apiVersion (string). Value aiven.io/v1alpha1.kind (string). Value Project.metadata (object). Data that identifies the object, including a name string and optional namespace.spec (object). ProjectSpec defines the desired state of Project. See below for nested schema.Appears on Project.
ProjectSpec defines the desired state of Project.
Optional
accountId (string, MaxLength: 32). Account ID.authSecretRef (object). Authentication reference to Aiven token in a secret. See below for nested schema.billingAddress (string, MaxLength: 1000). Billing name and address of the project.billingCurrency (string, Enum: AUD, CAD, CHF, DKK, EUR, GBP, NOK, SEK, USD). Billing currency.billingEmails (array of strings, MaxItems: 10). Billing contact emails of the project.billingExtraText (string, MaxLength: 1000). Extra text to be included in all project invoices, e.g. purchase order or cost center number.billingGroupId (string, Immutable, MinLength: 36, MaxLength: 36). BillingGroup ID.cardId (string, MaxLength: 64). Credit card ID; The ID may be either last 4 digits of the card or the actual ID.cloud (string, MaxLength: 256). Target cloud, example: aws-eu-central-1.connInfoSecretTarget (object). Secret configuration. See below for nested schema.connInfoSecretTargetDisabled (boolean, Immutable). When true, the secret containing connection information will not be created, defaults to false. This field cannot be changed after resource creation.copyFromProject (string, Immutable, MaxLength: 63). Project name from which to copy settings to the new project.countryCode (string, MinLength: 2, MaxLength: 2). Billing country code of the project.tags (object, AdditionalProperties: string). Tags are key-value pairs that allow you to categorize projects.technicalEmails (array of strings, MaxItems: 10). Technical contact emails of the project.Appears on spec.
Authentication reference to Aiven token in a secret.
Required
key (string, MinLength: 1).name (string, MinLength: 1).Appears on spec.
Secret configuration.
Required
name (string, Immutable). Name of the secret resource to be created. By default, it is equal to the resource name.Optional
annotations (object, AdditionalProperties: string). Annotations added to the secret.labels (object, AdditionalProperties: string). Labels added to the secret.prefix (string). Prefix for the secret's keys. Added \"as is\" without any transformations. By default, is equal to the kind name in uppercase + underscore, e.g. KAFKA_, REDIS_, etc.apiVersion: aiven.io/v1alpha1\nkind: ProjectVPC\nmetadata:\n name: my-project-vpc\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n project: aiven-project-name\n cloudName: google-europe-west1\n networkCidr: 10.0.0.0/24\nInfo
To create this resource, a Secret containing Aiven token must be created first.
Apply the resource with:
kubectl apply -f example.yaml\nVerify the newly created ProjectVPC:
kubectl get projectvpcs my-project-vpc\nThe output is similar to the following:
Name Project Cloud Network CIDR State \nmy-project-vpc aiven-project-name google-europe-west1 10.0.0.0/24 RUNNING \nProjectVPC is the Schema for the projectvpcs API.
Required
apiVersion (string). Value aiven.io/v1alpha1.kind (string). Value ProjectVPC.metadata (object). Data that identifies the object, including a name string and optional namespace.spec (object). ProjectVPCSpec defines the desired state of ProjectVPC. See below for nested schema.Appears on ProjectVPC.
ProjectVPCSpec defines the desired state of ProjectVPC.
Required
cloudName (string, Immutable, MaxLength: 256). Cloud the VPC is in.networkCidr (string, Immutable, MaxLength: 36). Network address range used by the VPC like 192.168.0.0/24.project (string, Immutable, Pattern: ^[a-zA-Z0-9_-]+$, MaxLength: 63). Identifies the project this resource belongs to.Optional
authSecretRef (object). Authentication reference to Aiven token in a secret. See below for nested schema.Appears on spec.
Authentication reference to Aiven token in a secret.
Required
key (string, MinLength: 1).name (string, MinLength: 1).apiVersion: aiven.io/v1alpha1\nkind: Redis\nmetadata:\n name: k8s-redis\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n connInfoSecretTarget:\n name: redis-token\n prefix: MY_SECRET_PREFIX_\n annotations:\n foo: bar\n labels:\n baz: egg\n\n project: my-aiven-project\n cloudName: google-europe-west1\n plan: startup-4\n\n maintenanceWindowDow: friday\n maintenanceWindowTime: 23:00:00\n\n userConfig:\n redis_maxmemory_policy: allkeys-random\nInfo
To create this resource, a Secret containing Aiven token must be created first.
Apply the resource with:
kubectl apply -f example.yaml\nVerify the newly created Redis:
kubectl get redis k8s-redis\nThe output is similar to the following:
Name Project Region Plan State \nk8s-redis my-aiven-project google-europe-west1 startup-4 RUNNING \nTo view the details of the Secret, use the following command:
kubectl describe secret redis-token\nYou can use the jq to quickly decode the Secret:
kubectl get secret redis-token -o json | jq '.data | map_values(@base64d)'\nThe output is similar to the following:
{\n \"REDIS_HOST\": \"<secret>\",\n \"REDIS_PORT\": \"<secret>\",\n \"REDIS_USER\": \"<secret>\",\n \"REDIS_PASSWORD\": \"<secret>\",\n}\nRedis is the Schema for the redis API.
Exposes secret keys
REDIS_HOST, REDIS_PORT, REDIS_USER, REDIS_PASSWORD.
Required
apiVersion (string). Value aiven.io/v1alpha1.kind (string). Value Redis.metadata (object). Data that identifies the object, including a name string and optional namespace.spec (object). RedisSpec defines the desired state of Redis. See below for nested schema.Appears on Redis.
RedisSpec defines the desired state of Redis.
Required
plan (string, MaxLength: 128). Subscription plan.project (string, Immutable, Pattern: ^[a-zA-Z0-9_-]+$, MaxLength: 63). Identifies the project this resource belongs to.Optional
authSecretRef (object). Authentication reference to Aiven token in a secret. See below for nested schema.cloudName (string, MaxLength: 256). Cloud the service runs in.connInfoSecretTarget (object). Secret configuration. See below for nested schema.connInfoSecretTargetDisabled (boolean, Immutable). When true, the secret containing connection information will not be created, defaults to false. This field cannot be changed after resource creation.disk_space (string, Pattern: (?i)^[1-9][0-9]*(GiB|G)?$). The disk space of the service, possible values depend on the service type, the cloud provider and the project. Reducing will result in the service re-balancing. The removal of this field does not change the value.maintenanceWindowDow (string, Enum: monday, tuesday, wednesday, thursday, friday, saturday, sunday). Day of week when maintenance operations should be performed. One monday, tuesday, wednesday, etc.maintenanceWindowTime (string, MaxLength: 8). Time of day when maintenance operations should be performed. UTC time in HH:mm:ss format.projectVPCRef (object). ProjectVPCRef reference to ProjectVPC resource to use its ID as ProjectVPCID automatically. See below for nested schema.projectVpcId (string, MaxLength: 36). Identifier of the VPC the service should be in, if any.serviceIntegrations (array of objects, Immutable, MaxItems: 1). Service integrations to specify when creating a service. Not applied after initial service creation. See below for nested schema.tags (object, AdditionalProperties: string). Tags are key-value pairs that allow you to categorize services.technicalEmails (array of objects, MaxItems: 10). Defines the email addresses that will receive alerts about upcoming maintenance updates or warnings about service instability. See below for nested schema.terminationProtection (boolean). Prevent service from being deleted. It is recommended to have this enabled for all services.userConfig (object). Redis specific user configuration options. See below for nested schema.Appears on spec.
Authentication reference to Aiven token in a secret.
Required
key (string, MinLength: 1).name (string, MinLength: 1).Appears on spec.
Secret configuration.
Required
name (string, Immutable). Name of the secret resource to be created. By default, it is equal to the resource name.Optional
annotations (object, AdditionalProperties: string). Annotations added to the secret.labels (object, AdditionalProperties: string). Labels added to the secret.prefix (string). Prefix for the secret's keys. Added \"as is\" without any transformations. By default, is equal to the kind name in uppercase + underscore, e.g. KAFKA_, REDIS_, etc.Appears on spec.
ProjectVPCRef reference to ProjectVPC resource to use its ID as ProjectVPCID automatically.
Required
name (string, MinLength: 1).Optional
namespace (string, MinLength: 1).Appears on spec.
Service integrations to specify when creating a service. Not applied after initial service creation.
Required
integrationType (string, Enum: read_replica).sourceServiceName (string, MinLength: 1, MaxLength: 64).Appears on spec.
Defines the email addresses that will receive alerts about upcoming maintenance updates or warnings about service instability.
Required
email (string). Email address.Appears on spec.
Redis specific user configuration options.
Optional
additional_backup_regions (array of strings, MaxItems: 1). Additional Cloud Regions for Backup Replication.ip_filter (array of objects, MaxItems: 1024). Allow incoming connections from CIDR address block, e.g. 10.20.0.0/16. See below for nested schema.migration (object). Migrate data from existing server. See below for nested schema.private_access (object). Allow access to selected service ports from private networks. See below for nested schema.privatelink_access (object). Allow access to selected service components through Privatelink. See below for nested schema.project_to_fork_from (string, Immutable, Pattern: ^[a-z][-a-z0-9]{0,63}$|^$, MaxLength: 63). Name of another project to fork a service from. This has effect only when a new service is being created.public_access (object). Allow access to selected service ports from the public Internet. See below for nested schema.recovery_basebackup_name (string, Pattern: ^[a-zA-Z0-9-_:.]+$, MaxLength: 128). Name of the basebackup to restore in forked service.redis_acl_channels_default (string, Enum: allchannels, resetchannels). Determines default pub/sub channels' ACL for new users if ACL is not supplied. When this option is not defined, all_channels is assumed to keep backward compatibility. This option doesn't affect Redis configuration acl-pubsub-default.redis_io_threads (integer, Minimum: 1, Maximum: 32). Set Redis IO thread count. Changing this will cause a restart of the Redis service.redis_lfu_decay_time (integer, Minimum: 1, Maximum: 120). LFU maxmemory-policy counter decay time in minutes.redis_lfu_log_factor (integer, Minimum: 0, Maximum: 100). Counter logarithm factor for volatile-lfu and allkeys-lfu maxmemory-policies.redis_maxmemory_policy (string, Enum: noeviction, allkeys-lru, volatile-lru, allkeys-random, volatile-random, volatile-ttl, volatile-lfu, allkeys-lfu). Redis maxmemory-policy.redis_notify_keyspace_events (string, Pattern: ^[KEg\\$lshzxentdmA]*$, MaxLength: 32). Set notify-keyspace-events option.redis_number_of_databases (integer, Minimum: 1, Maximum: 128). Set number of Redis databases. Changing this will cause a restart of the Redis service.redis_persistence (string, Enum: off, rdb). When persistence is rdb, Redis does RDB dumps each 10 minutes if any key is changed. Also RDB dumps are done according to the backup schedule for backup purposes. When persistence is off, no RDB dumps or backups are done, so data can be lost at any moment if the service is restarted for any reason, or if the service is powered off. Also, the service can't be forked.redis_pubsub_client_output_buffer_limit (integer, Minimum: 32, Maximum: 512). Set output buffer limit for pub / sub clients in MB. The value is the hard limit, the soft limit is 1/4 of the hard limit. When setting the limit, be mindful of the available memory in the selected service plan.redis_ssl (boolean). Require SSL to access Redis.redis_timeout (integer, Minimum: 0, Maximum: 31536000). Redis idle connection timeout in seconds.redis_version (string, Enum: 7.0). Redis major version.service_log (boolean). Store logs for the service so that they are available in the HTTP API and console.service_to_fork_from (string, Immutable, Pattern: ^[a-z][-a-z0-9]{0,63}$|^$, MaxLength: 64). Name of another service to fork from. This has effect only when a new service is being created.static_ips (boolean). Use static public IP addresses.Appears on spec.userConfig.
CIDR address block, either as a string, or in a dict with an optional description field.
Required
network (string, MaxLength: 43). CIDR address block.Optional
description (string, MaxLength: 1024). Description for IP filter list entry.Appears on spec.userConfig.
Migrate data from existing server.
Required
host (string, MaxLength: 255). Hostname or IP address of the server where to migrate data from.port (integer, Minimum: 1, Maximum: 65535). Port number of the server where to migrate data from.Optional
dbname (string, MaxLength: 63). Database name for bootstrapping the initial connection.ignore_dbs (string, MaxLength: 2048). Comma-separated list of databases, which should be ignored during migration (supported by MySQL and PostgreSQL only at the moment).method (string, Enum: dump, replication). The migration method to be used (currently supported only by Redis, Dragonfly, MySQL and PostgreSQL service types).password (string, MaxLength: 256). Password for authentication with the server where to migrate data from.ssl (boolean). The server where to migrate data from is secured with SSL.username (string, MaxLength: 256). User name for authentication with the server where to migrate data from.Appears on spec.userConfig.
Allow access to selected service ports from private networks.
Optional
prometheus (boolean). Allow clients to connect to prometheus with a DNS name that always resolves to the service's private IP addresses. Only available in certain network locations.redis (boolean). Allow clients to connect to redis with a DNS name that always resolves to the service's private IP addresses. Only available in certain network locations.Appears on spec.userConfig.
Allow access to selected service components through Privatelink.
Optional
prometheus (boolean). Enable prometheus.redis (boolean). Enable redis.Appears on spec.userConfig.
Allow access to selected service ports from the public Internet.
Optional
prometheus (boolean). Allow clients to connect to prometheus from the public internet for service nodes that are in a project VPC or another type of private network.redis (boolean). Allow clients to connect to redis from the public internet for service nodes that are in a project VPC or another type of private network.apiVersion: aiven.io/v1alpha1\nkind: ServiceIntegration\nmetadata:\n name: my-service-integration\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n project: aiven-project-name\n integrationType: clickhouse_postgresql\n sourceServiceName: my-pg\n destinationServiceName: my-clickhouse\n\n clickhousePostgresql:\n databases:\n - database: defaultdb\n schema: public\n\n---\n\napiVersion: aiven.io/v1alpha1\nkind: Clickhouse\nmetadata:\n name: my-clickhouse\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n project: aiven-project-name\n cloudName: google-europe-west1\n plan: startup-16\n maintenanceWindowDow: friday\n maintenanceWindowTime: 23:00:00\n\n---\n\napiVersion: aiven.io/v1alpha1\nkind: PostgreSQL\nmetadata:\n name: my-pg\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n project: aiven-project-name\n cloudName: google-europe-west1\n plan: startup-4\n maintenanceWindowDow: friday\n maintenanceWindowTime: 23:00:00\napiVersion: aiven.io/v1alpha1\nkind: ServiceIntegration\nmetadata:\n name: my-service-integration\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n project: aiven-project-name\n integrationType: datadog\n sourceServiceName: my-pg\n destinationEndpointId: destination-endpoint-id\n\n datadog:\n datadog_dbm_enabled: True\n datadog_tags:\n - tag: env\n comment: test\n\n---\n\napiVersion: aiven.io/v1alpha1\nkind: PostgreSQL\nmetadata:\n name: my-pg\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n project: aiven-project-name\n cloudName: google-europe-west1\n plan: startup-4\napiVersion: aiven.io/v1alpha1\nkind: ServiceIntegration\nmetadata:\n name: my-service-integration\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n project: aiven-project-name\n integrationType: kafka_connect\n sourceServiceName: my-kafka\n destinationServiceName: my-kafka-connect\n\n kafkaConnect:\n kafka_connect:\n group_id: connect\n status_storage_topic: __connect_status\n offset_storage_topic: __connect_offsets\n\n---\n\napiVersion: aiven.io/v1alpha1\nkind: Kafka\nmetadata:\n name: my-kafka\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n project: aiven-project-name\n cloudName: google-europe-west1\n plan: business-4\n\n---\n\napiVersion: aiven.io/v1alpha1\nkind: KafkaConnect\nmetadata:\n name: my-kafka-connect\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n project: aiven-project-name\n cloudName: google-europe-west1\n plan: business-4\n\n userConfig:\n kafka_connect:\n consumer_isolation_level: read_committed\n public_access:\n kafka_connect: true\napiVersion: aiven.io/v1alpha1\nkind: ServiceIntegration\nmetadata:\n name: my-service-integration\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n project: aiven-project-name\n integrationType: kafka_logs\n sourceServiceName: my-kafka\n destinationServiceName: my-kafka\n\n kafkaLogs:\n kafka_topic: my-kafka-topic\n\n---\n\napiVersion: aiven.io/v1alpha1\nkind: Kafka\nmetadata:\n name: my-kafka\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n project: aiven-project-name\n cloudName: google-europe-west1\n plan: business-4\n\n---\n\napiVersion: aiven.io/v1alpha1\nkind: KafkaTopic\nmetadata:\n name: my-kafka-topic\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n project: aiven-project-name\n serviceName: my-kafka\n replication: 2\n partitions: 1\nInfo
To create this resource, a Secret containing Aiven token must be created first.
Apply the resource with:
kubectl apply -f example.yaml\nVerify the newly created ServiceIntegration:
kubectl get serviceintegrations my-service-integration\nThe output is similar to the following:
Name Project Type Source Service Name Destination Service Name \nmy-service-integration aiven-project-name clickhouse_postgresql my-pg my-clickhouse \nServiceIntegration is the Schema for the serviceintegrations API.
Required
apiVersion (string). Value aiven.io/v1alpha1.kind (string). Value ServiceIntegration.metadata (object). Data that identifies the object, including a name string and optional namespace.spec (object). ServiceIntegrationSpec defines the desired state of ServiceIntegration. See below for nested schema.Appears on ServiceIntegration.
ServiceIntegrationSpec defines the desired state of ServiceIntegration.
Required
integrationType (string, Enum: alertmanager, autoscaler, caching, cassandra_cross_service_cluster, clickhouse_kafka, clickhouse_postgresql, dashboard, datadog, datasource, external_aws_cloudwatch_logs, external_aws_cloudwatch_metrics, external_elasticsearch_logs, external_google_cloud_logging, external_opensearch_logs, flink, flink_external_kafka, flink_external_postgresql, internal_connectivity, jolokia, kafka_connect, kafka_logs, kafka_mirrormaker, logs, m3aggregator, m3coordinator, metrics, opensearch_cross_cluster_replication, opensearch_cross_cluster_search, prometheus, read_replica, rsyslog, schema_registry_proxy, stresstester, thanosquery, thanosstore, vmalert, Immutable). Type of the service integration accepted by Aiven API. Some values may not be supported by the operator.project (string, Immutable, Pattern: ^[a-zA-Z0-9_-]+$, MaxLength: 63). Identifies the project this resource belongs to.Optional
authSecretRef (object). Authentication reference to Aiven token in a secret. See below for nested schema.clickhouseKafka (object). Clickhouse Kafka configuration values. See below for nested schema.clickhousePostgresql (object). Clickhouse PostgreSQL configuration values. See below for nested schema.datadog (object). Datadog specific user configuration options. See below for nested schema.destinationEndpointId (string, Immutable, MaxLength: 36). Destination endpoint for the integration (if any).destinationProjectName (string, Immutable, MaxLength: 63). Destination project for the integration (if any).destinationServiceName (string, Immutable, MaxLength: 64). Destination service for the integration (if any).externalAWSCloudwatchMetrics (object). External AWS CloudWatch Metrics integration Logs configuration values. See below for nested schema.kafkaConnect (object). Kafka Connect service configuration values. See below for nested schema.kafkaLogs (object). Kafka logs configuration values. See below for nested schema.kafkaMirrormaker (object). Kafka MirrorMaker configuration values. See below for nested schema.logs (object). Logs configuration values. See below for nested schema.metrics (object). Metrics configuration values. See below for nested schema.sourceEndpointID (string, Immutable, MaxLength: 36). Source endpoint for the integration (if any).sourceProjectName (string, Immutable, MaxLength: 63). Source project for the integration (if any).sourceServiceName (string, Immutable, MaxLength: 64). Source service for the integration (if any).Appears on spec.
Authentication reference to Aiven token in a secret.
Required
key (string, MinLength: 1).name (string, MinLength: 1).Appears on spec.
Clickhouse Kafka configuration values.
Required
tables (array of objects, MaxItems: 100). Tables to create. See below for nested schema.Appears on spec.clickhouseKafka.
Table to create.
Required
columns (array of objects, MaxItems: 100). Table columns. See below for nested schema.data_format (string, Enum: Avro, CSV, JSONAsString, JSONCompactEachRow, JSONCompactStringsEachRow, JSONEachRow, JSONStringsEachRow, MsgPack, TSKV, TSV, TabSeparated, RawBLOB, AvroConfluent, Parquet). Message data format.group_name (string, MinLength: 1, MaxLength: 249). Kafka consumers group.name (string, MinLength: 1, MaxLength: 40). Name of the table.topics (array of objects, MaxItems: 100). Kafka topics. See below for nested schema.Optional
auto_offset_reset (string, Enum: smallest, earliest, beginning, largest, latest, end). Action to take when there is no initial offset in offset store or the desired offset is out of range.date_time_input_format (string, Enum: basic, best_effort, best_effort_us). Method to read DateTime from text input formats.handle_error_mode (string, Enum: default, stream). How to handle errors for Kafka engine.max_block_size (integer, Minimum: 0, Maximum: 1000000000). Number of row collected by poll(s) for flushing data from Kafka.max_rows_per_message (integer, Minimum: 1, Maximum: 1000000000). The maximum number of rows produced in one kafka message for row-based formats.num_consumers (integer, Minimum: 1, Maximum: 10). The number of consumers per table per replica.poll_max_batch_size (integer, Minimum: 0, Maximum: 1000000000). Maximum amount of messages to be polled in a single Kafka poll.skip_broken_messages (integer, Minimum: 0, Maximum: 1000000000). Skip at least this number of broken messages from Kafka topic per block.Appears on spec.clickhouseKafka.tables.
Table column.
Required
name (string, MinLength: 1, MaxLength: 40). Column name.type (string, MinLength: 1, MaxLength: 1000). Column type.Appears on spec.clickhouseKafka.tables.
Kafka topic.
Required
name (string, MinLength: 1, MaxLength: 249). Name of the topic.Appears on spec.
Clickhouse PostgreSQL configuration values.
Required
databases (array of objects, MaxItems: 10). Databases to expose. See below for nested schema.Appears on spec.clickhousePostgresql.
Database to expose.
Optional
database (string, MinLength: 1, MaxLength: 63). PostgreSQL database to expose.schema (string, MinLength: 1, MaxLength: 63). PostgreSQL schema to expose.Appears on spec.
Datadog specific user configuration options.
Optional
datadog_dbm_enabled (boolean). Enable Datadog Database Monitoring.datadog_pgbouncer_enabled (boolean). Enable Datadog PgBouncer Metric Tracking.datadog_tags (array of objects, MaxItems: 32). Custom tags provided by user. See below for nested schema.exclude_consumer_groups (array of strings, MaxItems: 1024). List of custom metrics.exclude_topics (array of strings, MaxItems: 1024). List of topics to exclude.include_consumer_groups (array of strings, MaxItems: 1024). List of custom metrics.include_topics (array of strings, MaxItems: 1024). List of topics to include.kafka_custom_metrics (array of strings, MaxItems: 1024). List of custom metrics.max_jmx_metrics (integer, Minimum: 10, Maximum: 100000). Maximum number of JMX metrics to send.mirrormaker_custom_metrics (array of strings, MaxItems: 1024). List of custom metrics.opensearch (object). Datadog Opensearch Options. See below for nested schema.redis (object). Datadog Redis Options. See below for nested schema.Appears on spec.datadog.
Datadog tag defined by user.
Required
tag (string, MinLength: 1, MaxLength: 200). Tag format and usage are described here: https://docs.datadoghq.com/getting_started/tagging. Tags with prefix aiven- are reserved for Aiven.Optional
comment (string, MaxLength: 1024). Optional tag explanation.Appears on spec.datadog.
Datadog Opensearch Options.
Optional
cluster_stats_enabled (boolean). Enable Datadog Opensearch Cluster Monitoring.index_stats_enabled (boolean). Enable Datadog Opensearch Index Monitoring.pending_task_stats_enabled (boolean). Enable Datadog Opensearch Pending Task Monitoring.pshard_stats_enabled (boolean). Enable Datadog Opensearch Primary Shard Monitoring.Appears on spec.datadog.
Datadog Redis Options.
Required
command_stats_enabled (boolean). Enable command_stats option in the agent's configuration.Appears on spec.
External AWS CloudWatch Metrics integration Logs configuration values.
Optional
dropped_metrics (array of objects, MaxItems: 1024). Metrics to not send to AWS CloudWatch (takes precedence over extra_metrics). See below for nested schema.extra_metrics (array of objects, MaxItems: 1024). Metrics to allow through to AWS CloudWatch (in addition to default metrics). See below for nested schema.Appears on spec.externalAWSCloudwatchMetrics.
Metric name and subfield.
Required
field (string, MaxLength: 1000). Identifier of a value in the metric.metric (string, MaxLength: 1000). Identifier of the metric.Appears on spec.externalAWSCloudwatchMetrics.
Metric name and subfield.
Required
field (string, MaxLength: 1000). Identifier of a value in the metric.metric (string, MaxLength: 1000). Identifier of the metric.Appears on spec.
Kafka Connect service configuration values.
Required
kafka_connect (object). Kafka Connect service configuration values. See below for nested schema.Appears on spec.kafkaConnect.
Kafka Connect service configuration values.
Optional
config_storage_topic (string, MaxLength: 249). The name of the topic where connector and task configuration data are stored.This must be the same for all workers with the same group_id.group_id (string, MaxLength: 249). A unique string that identifies the Connect cluster group this worker belongs to.offset_storage_topic (string, MaxLength: 249). The name of the topic where connector and task configuration offsets are stored.This must be the same for all workers with the same group_id.status_storage_topic (string, MaxLength: 249). The name of the topic where connector and task configuration status updates are stored.This must be the same for all workers with the same group_id.Appears on spec.
Kafka logs configuration values.
Required
kafka_topic (string, MinLength: 1, MaxLength: 249). Topic name.Optional
selected_log_fields (array of strings, MaxItems: 5). The list of logging fields that will be sent to the integration logging service. The MESSAGE and timestamp fields are always sent.Appears on spec.
Kafka MirrorMaker configuration values.
Optional
cluster_alias (string, Pattern: ^[a-zA-Z0-9_.-]+$, MaxLength: 128). The alias under which the Kafka cluster is known to MirrorMaker. Can contain the following symbols: ASCII alphanumerics, ., _, and -.kafka_mirrormaker (object). Kafka MirrorMaker configuration values. See below for nested schema.Appears on spec.kafkaMirrormaker.
Kafka MirrorMaker configuration values.
Optional
consumer_auto_offset_reset (string, Enum: earliest, latest). Set where consumer starts to consume data. Value earliest: Start replication from the earliest offset. Value latest: Start replication from the latest offset. Default is earliest.consumer_fetch_min_bytes (integer, Minimum: 1, Maximum: 5242880). The minimum amount of data the server should return for a fetch request.consumer_max_poll_records (integer, Minimum: 100, Maximum: 20000). Set consumer max.poll.records. The default is 500.producer_batch_size (integer, Minimum: 0, Maximum: 5242880). The batch size in bytes producer will attempt to collect before publishing to broker.producer_buffer_memory (integer, Minimum: 5242880, Maximum: 134217728). The amount of bytes producer can use for buffering data before publishing to broker.producer_compression_type (string, Enum: gzip, snappy, lz4, zstd, none). Specify the default compression type for producers. This configuration accepts the standard compression codecs (gzip, snappy, lz4, zstd). It additionally accepts none which is the default and equivalent to no compression.producer_linger_ms (integer, Minimum: 0, Maximum: 5000). The linger time (ms) for waiting new data to arrive for publishing.producer_max_request_size (integer, Minimum: 0, Maximum: 268435456). The maximum request size in bytes.Appears on spec.
Logs configuration values.
Optional
elasticsearch_index_days_max (integer, Minimum: 1, Maximum: 10000). Elasticsearch index retention limit.elasticsearch_index_prefix (string, MinLength: 1, MaxLength: 1024). Elasticsearch index prefix.selected_log_fields (array of strings, MaxItems: 5). The list of logging fields that will be sent to the integration logging service. The MESSAGE and timestamp fields are always sent.Appears on spec.
Metrics configuration values.
Optional
database (string, Pattern: ^[_A-Za-z0-9][-_A-Za-z0-9]{0,39}$, MaxLength: 40). Name of the database where to store metric datapoints. Only affects PostgreSQL destinations. Defaults to metrics. Note that this must be the same for all metrics integrations that write data to the same PostgreSQL service.retention_days (integer, Minimum: 0, Maximum: 10000). Number of days to keep old metrics. Only affects PostgreSQL destinations. Set to 0 for no automatic cleanup. Defaults to 30 days.ro_username (string, Pattern: ^[_A-Za-z0-9][-._A-Za-z0-9]{0,39}$, MaxLength: 40). Name of a user that can be used to read metrics. This will be used for Grafana integration (if enabled) to prevent Grafana users from making undesired changes. Only affects PostgreSQL destinations. Defaults to metrics_reader. Note that this must be the same for all metrics integrations that write data to the same PostgreSQL service.source_mysql (object). Configuration options for metrics where source service is MySQL. See below for nested schema.username (string, Pattern: ^[_A-Za-z0-9][-._A-Za-z0-9]{0,39}$, MaxLength: 40). Name of the user used to write metrics. Only affects PostgreSQL destinations. Defaults to metrics_writer. Note that this must be the same for all metrics integrations that write data to the same PostgreSQL service.Appears on spec.metrics.
Configuration options for metrics where source service is MySQL.
Required
telegraf (object). Configuration options for Telegraf MySQL input plugin. See below for nested schema.Appears on spec.metrics.source_mysql.
Configuration options for Telegraf MySQL input plugin.
Optional
gather_event_waits (boolean). Gather metrics from PERFORMANCE_SCHEMA.EVENT_WAITS.gather_file_events_stats (boolean). gather metrics from PERFORMANCE_SCHEMA.FILE_SUMMARY_BY_EVENT_NAME.gather_index_io_waits (boolean). Gather metrics from PERFORMANCE_SCHEMA.TABLE_IO_WAITS_SUMMARY_BY_INDEX_USAGE.gather_info_schema_auto_inc (boolean). Gather auto_increment columns and max values from information schema.gather_innodb_metrics (boolean). Gather metrics from INFORMATION_SCHEMA.INNODB_METRICS.gather_perf_events_statements (boolean). Gather metrics from PERFORMANCE_SCHEMA.EVENTS_STATEMENTS_SUMMARY_BY_DIGEST.gather_process_list (boolean). Gather thread state counts from INFORMATION_SCHEMA.PROCESSLIST.gather_slave_status (boolean). Gather metrics from SHOW SLAVE STATUS command output.gather_table_io_waits (boolean). Gather metrics from PERFORMANCE_SCHEMA.TABLE_IO_WAITS_SUMMARY_BY_TABLE.gather_table_lock_waits (boolean). Gather metrics from PERFORMANCE_SCHEMA.TABLE_LOCK_WAITS.gather_table_schema (boolean). Gather metrics from INFORMATION_SCHEMA.TABLES.perf_events_statements_digest_text_limit (integer, Minimum: 1, Maximum: 2048). Truncates digest text from perf_events_statements into this many characters.perf_events_statements_limit (integer, Minimum: 1, Maximum: 4000). Limits metrics from perf_events_statements.perf_events_statements_time_limit (integer, Minimum: 1, Maximum: 2592000). Only include perf_events_statements whose last seen is less than this many seconds.apiVersion: aiven.io/v1alpha1\nkind: ServiceIntegrationEndpoint\nmetadata:\n name: my-service-integration-endpoint\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n project: aiven-project-name\n endpointName: my-external-postgresql\n endpointType: external_postgresql\n\n externalPostgresql:\n username: username\n password: password\n host: example.example\n port: 5432\n ssl_mode: require\napiVersion: aiven.io/v1alpha1\nkind: ServiceIntegrationEndpoint\nmetadata:\n name: my-service-integration-endpoint\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n project: aiven-project-name\n endpointName: my-external-schema-registry\n endpointType: external_schema_registry\n\n externalSchemaRegistry:\n url: https://schema-registry.example.com:8081\n authentication: basic\n basic_auth_username: username\n basic_auth_password: password\nInfo
To create this resource, a Secret containing Aiven token must be created first.
Apply the resource with:
kubectl apply -f example.yaml\nVerify the newly created ServiceIntegrationEndpoint:
kubectl get serviceintegrationendpoints my-service-integration-endpoint\nThe output is similar to the following:
Name Project Endpoint Name Endpoint Type ID \nmy-service-integration-endpoint aiven-project-name my-external-postgresql external_postgresql <id> \nServiceIntegrationEndpoint is the Schema for the serviceintegrationendpoints API.
Required
apiVersion (string). Value aiven.io/v1alpha1.kind (string). Value ServiceIntegrationEndpoint.metadata (object). Data that identifies the object, including a name string and optional namespace.spec (object). ServiceIntegrationEndpointSpec defines the desired state of ServiceIntegrationEndpoint. See below for nested schema.Appears on ServiceIntegrationEndpoint.
ServiceIntegrationEndpointSpec defines the desired state of ServiceIntegrationEndpoint.
Required
endpointType (string, Enum: autoscaler, datadog, external_aws_cloudwatch_logs, external_aws_cloudwatch_metrics, external_aws_s3, external_clickhouse, external_elasticsearch_logs, external_google_cloud_bigquery, external_google_cloud_logging, external_kafka, external_mysql, external_opensearch_logs, external_postgresql, external_redis, external_schema_registry, external_sumologic_logs, jolokia, prometheus, rsyslog, Immutable). Type of the service integration endpoint.project (string, Immutable, Pattern: ^[a-zA-Z0-9_-]+$, MaxLength: 63). Identifies the project this resource belongs to.Optional
authSecretRef (object). Authentication reference to Aiven token in a secret. See below for nested schema.datadog (object). Datadog configuration values. See below for nested schema.endpointName (string, Immutable, MaxLength: 36). Source endpoint for the integration (if any).externalAWSCloudwatchLogs (object). ExternalAwsCloudwatchLogs configuration values. See below for nested schema.externalAWSCloudwatchMetrics (object). ExternalAwsCloudwatchMetrics configuration values. See below for nested schema.externalElasticsearchLogs (object). ExternalElasticsearchLogs configuration values. See below for nested schema.externalGoogleCloudBigquery (object). ExternalGoogleCloudBigquery configuration values. See below for nested schema.externalGoogleCloudLogging (object). ExternalGoogleCloudLogging configuration values. See below for nested schema.externalKafka (object). ExternalKafka configuration values. See below for nested schema.externalOpensearchLogs (object). ExternalOpensearchLogs configuration values. See below for nested schema.externalPostgresql (object). ExternalPostgresql configuration values. See below for nested schema.externalSchemaRegistry (object). ExternalSchemaRegistry configuration values. See below for nested schema.jolokia (object). Jolokia configuration values. See below for nested schema.prometheus (object). Prometheus configuration values. See below for nested schema.rsyslog (object). Rsyslog configuration values. See below for nested schema.Appears on spec.
Authentication reference to Aiven token in a secret.
Required
key (string, MinLength: 1).name (string, MinLength: 1).Appears on spec.
Datadog configuration values.
Required
datadog_api_key (string, Pattern: ^[A-Za-z0-9]{1,256}$, MinLength: 1, MaxLength: 256). Datadog API key.Optional
datadog_tags (array of objects, MaxItems: 32). Custom tags provided by user. See below for nested schema.disable_consumer_stats (boolean). Disable consumer group metrics.kafka_consumer_check_instances (integer, Minimum: 1, Maximum: 100). Number of separate instances to fetch kafka consumer statistics with.kafka_consumer_stats_timeout (integer, Minimum: 2, Maximum: 300). Number of seconds that datadog will wait to get consumer statistics from brokers.max_partition_contexts (integer, Minimum: 200, Maximum: 200000). Maximum number of partition contexts to send.site (string, Enum: datadoghq.com, datadoghq.eu, us3.datadoghq.com, us5.datadoghq.com, ddog-gov.com, ap1.datadoghq.com). Datadog intake site. Defaults to datadoghq.com.Appears on spec.datadog.
Datadog tag defined by user.
Required
tag (string, MinLength: 1, MaxLength: 200). Tag format and usage are described here: https://docs.datadoghq.com/getting_started/tagging. Tags with prefix aiven- are reserved for Aiven.Optional
comment (string, MaxLength: 1024). Optional tag explanation.Appears on spec.
ExternalAwsCloudwatchLogs configuration values.
Required
access_key (string, MaxLength: 4096). AWS access key. Required permissions are logs:CreateLogGroup, logs:CreateLogStream, logs:PutLogEvents and logs:DescribeLogStreams.region (string, MaxLength: 32). AWS region.secret_key (string, MaxLength: 4096). AWS secret key.Optional
log_group_name (string, Pattern: ^[\\.\\-_/#A-Za-z0-9]+$, MinLength: 1, MaxLength: 512). AWS CloudWatch log group name.Appears on spec.
ExternalAwsCloudwatchMetrics configuration values.
Required
access_key (string, MaxLength: 4096). AWS access key. Required permissions are cloudwatch:PutMetricData.namespace (string, MinLength: 1, MaxLength: 255). AWS CloudWatch Metrics Namespace.region (string, MaxLength: 32). AWS region.secret_key (string, MaxLength: 4096). AWS secret key.Appears on spec.
ExternalElasticsearchLogs configuration values.
Required
index_prefix (string, Pattern: ^[a-z0-9][a-z0-9-_.]+$, MinLength: 1, MaxLength: 1000). Elasticsearch index prefix.url (string, MinLength: 12, MaxLength: 2048). Elasticsearch connection URL.Optional
ca (string, MaxLength: 16384). PEM encoded CA certificate.index_days_max (integer, Minimum: 1, Maximum: 10000). Maximum number of days of logs to keep.timeout (number, Minimum: 10, Maximum: 120). Elasticsearch request timeout limit.Appears on spec.
ExternalGoogleCloudBigquery configuration values.
Required
project_id (string, MinLength: 6, MaxLength: 30). GCP project id.service_account_credentials (string, MaxLength: 4096). This is a JSON object with the fields documented in https://cloud.google.com/iam/docs/creating-managing-service-account-keys .Appears on spec.
ExternalGoogleCloudLogging configuration values.
Required
log_id (string, MaxLength: 512). Google Cloud Logging log id.project_id (string, MinLength: 6, MaxLength: 30). GCP project id.service_account_credentials (string, MaxLength: 4096). This is a JSON object with the fields documented in https://cloud.google.com/iam/docs/creating-managing-service-account-keys .Appears on spec.
ExternalKafka configuration values.
Required
bootstrap_servers (string, MinLength: 3, MaxLength: 256). Bootstrap servers.security_protocol (string, Enum: PLAINTEXT, SSL, SASL_PLAINTEXT, SASL_SSL). Security protocol.Optional
sasl_mechanism (string, Enum: PLAIN, SCRAM-SHA-256, SCRAM-SHA-512). SASL mechanism used for connections to the Kafka server.sasl_plain_password (string, MinLength: 1, MaxLength: 256). Password for SASL PLAIN mechanism in the Kafka server.sasl_plain_username (string, MinLength: 1, MaxLength: 256). Username for SASL PLAIN mechanism in the Kafka server.ssl_ca_cert (string, MaxLength: 16384). PEM-encoded CA certificate.ssl_client_cert (string, MaxLength: 16384). PEM-encoded client certificate.ssl_client_key (string, MaxLength: 16384). PEM-encoded client key.ssl_endpoint_identification_algorithm (string, Enum: https). The endpoint identification algorithm to validate server hostname using server certificate.Appears on spec.
ExternalOpensearchLogs configuration values.
Required
index_prefix (string, Pattern: ^[a-z0-9][a-z0-9-_.]+$, MinLength: 1, MaxLength: 1000). OpenSearch index prefix.url (string, MinLength: 12, MaxLength: 2048). OpenSearch connection URL.Optional
ca (string, MaxLength: 16384). PEM encoded CA certificate.index_days_max (integer, Minimum: 1, Maximum: 10000). Maximum number of days of logs to keep.timeout (number, Minimum: 10, Maximum: 120). OpenSearch request timeout limit.Appears on spec.
ExternalPostgresql configuration values.
Required
host (string, MaxLength: 255). Hostname or IP address of the server.port (integer, Minimum: 1, Maximum: 65535). Port number of the server.username (string, MaxLength: 256). User name.Optional
default_database (string, Pattern: ^[_A-Za-z0-9][-_A-Za-z0-9]{0,62}$, MaxLength: 63). Default database.password (string, MaxLength: 256). Password.ssl_client_certificate (string, MaxLength: 16384). Client certificate.ssl_client_key (string, MaxLength: 16384). Client key.ssl_mode (string, Enum: require, verify-ca, verify-full). SSL mode to use for the connection. Please note that Aiven requires TLS for all connections to external PostgreSQL services.ssl_root_cert (string, MaxLength: 16384). SSL Root Cert.Appears on spec.
ExternalSchemaRegistry configuration values.
Required
authentication (string, Enum: none, basic). Authentication method.url (string, MaxLength: 2048). Schema Registry URL.Optional
basic_auth_password (string, MaxLength: 256). Basic authentication password.basic_auth_username (string, MaxLength: 256). Basic authentication user name.Appears on spec.
Jolokia configuration values.
Optional
basic_auth_password (string, MinLength: 8, MaxLength: 64). Jolokia basic authentication password.basic_auth_username (string, Pattern: ^[a-z0-9\\-@_]{5,32}$, MinLength: 5, MaxLength: 32). Jolokia basic authentication username.Appears on spec.
Prometheus configuration values.
Optional
basic_auth_password (string, MinLength: 8, MaxLength: 64). Prometheus basic authentication password.basic_auth_username (string, Pattern: ^[a-z0-9\\-@_]{5,32}$, MinLength: 5, MaxLength: 32). Prometheus basic authentication username.Appears on spec.
Rsyslog configuration values.
Required
format (string, Enum: rfc5424, rfc3164, custom). Message format.port (integer, Minimum: 1, Maximum: 65535). Rsyslog server port.server (string, MinLength: 4, MaxLength: 255). Rsyslog server IP address or hostname.tls (boolean). Require TLS.Optional
ca (string, MaxLength: 16384). PEM encoded CA certificate.cert (string, MaxLength: 16384). PEM encoded client certificate.key (string, MaxLength: 16384). PEM encoded client key.logline (string, Pattern: ^[ -~\\t]+$, MinLength: 1, MaxLength: 512). Custom syslog message format.max_message_size (integer, Minimum: 2048, Maximum: 2147483647). Rsyslog max message size.sd (string, MaxLength: 1024). Structured data block for log message.apiVersion: aiven.io/v1alpha1\nkind: ServiceUser\nmetadata:\n name: my-service-user\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n connInfoSecretTarget:\n name: service-user-secret\n prefix: MY_SECRET_PREFIX_\n annotations:\n foo: bar\n labels:\n baz: egg\n\n project: aiven-project-name\n serviceName: my-service-name\nInfo
To create this resource, a Secret containing Aiven token must be created first.
Apply the resource with:
kubectl apply -f example.yaml\nVerify the newly created ServiceUser:
kubectl get serviceusers my-service-user\nThe output is similar to the following:
Name Service Name Project \nmy-service-user my-service-name aiven-project-name \nTo view the details of the Secret, use the following command:
kubectl describe secret service-user-secret\nYou can use the jq to quickly decode the Secret:
kubectl get secret service-user-secret -o json | jq '.data | map_values(@base64d)'\nThe output is similar to the following:
{\n \"SERVICEUSER_HOST\": \"<secret>\",\n \"SERVICEUSER_PORT\": \"<secret>\",\n \"SERVICEUSER_USERNAME\": \"<secret>\",\n \"SERVICEUSER_PASSWORD\": \"<secret>\",\n \"SERVICEUSER_CA_CERT\": \"<secret>\",\n \"SERVICEUSER_ACCESS_CERT\": \"<secret>\",\n \"SERVICEUSER_ACCESS_KEY\": \"<secret>\",\n}\nServiceUser is the Schema for the serviceusers API.
Exposes secret keys
SERVICEUSER_HOST, SERVICEUSER_PORT, SERVICEUSER_USERNAME, SERVICEUSER_PASSWORD, SERVICEUSER_CA_CERT, SERVICEUSER_ACCESS_CERT, SERVICEUSER_ACCESS_KEY.
Required
apiVersion (string). Value aiven.io/v1alpha1.kind (string). Value ServiceUser.metadata (object). Data that identifies the object, including a name string and optional namespace.spec (object). ServiceUserSpec defines the desired state of ServiceUser. See below for nested schema.Appears on ServiceUser.
ServiceUserSpec defines the desired state of ServiceUser.
Required
project (string, Immutable, Pattern: ^[a-zA-Z0-9_-]+$, MaxLength: 63). Identifies the project this resource belongs to.serviceName (string, Immutable, Pattern: ^[a-z][-a-z0-9]+$, MaxLength: 63). Specifies the name of the service that this resource belongs to.Optional
authSecretRef (object). Authentication reference to Aiven token in a secret. See below for nested schema.authentication (string, Enum: caching_sha2_password, mysql_native_password). Authentication details.connInfoSecretTarget (object). Secret configuration. See below for nested schema.connInfoSecretTargetDisabled (boolean, Immutable). When true, the secret containing connection information will not be created, defaults to false. This field cannot be changed after resource creation.Appears on spec.
Authentication reference to Aiven token in a secret.
Required
key (string, MinLength: 1).name (string, MinLength: 1).Appears on spec.
Secret configuration.
Required
name (string, Immutable). Name of the secret resource to be created. By default, it is equal to the resource name.Optional
annotations (object, AdditionalProperties: string). Annotations added to the secret.labels (object, AdditionalProperties: string). Labels added to the secret.prefix (string). Prefix for the secret's keys. Added \"as is\" without any transformations. By default, is equal to the kind name in uppercase + underscore, e.g. KAFKA_, REDIS_, etc.The Aiven Operator for Kubernetes project accepts contributions via GitHub pull requests. This document outlines the process to help get your contribution accepted.
Please see also the Aiven Operator for Kubernetes Developer Guide.
"},{"location":"contributing/index.html#support-channels","title":"Support Channels","text":"This project offers support through GitHub issues and can be filed here. Moreover, GitHub issues are used as the primary method for tracking anything to do with the Aiven Operator for Kubernetes project.
"},{"location":"contributing/index.html#pull-request-process","title":"Pull Request Process","text":"In the interest of fostering an open and welcoming environment, we as contributors and maintainers pledge to making participation in our project and our community a harassment-free experience for everyone, regardless of age, body size, disability, ethnicity, gender identity and expression, level of experience, nationality, personal appearance, race, religion, or sexual identity and orientation.
"},{"location":"contributing/index.html#our-standards","title":"Our Standards","text":"Examples of behavior that contributes to creating a positive environment include:
Examples of unacceptable behavior by participants include:
This project adheres to the Conventional Commits specification. Please, make sure that your commit messages follow that specification.
"},{"location":"contributing/index.html#our-responsibilities","title":"Our Responsibilities","text":"Project maintainers are responsible for clarifying the standards of acceptable behavior and are expected to take appropriate and fair corrective action in response to any instances of unacceptable behavior.
Project maintainers have the right and responsibility to remove, edit, or reject comments, commits, code, wiki edits, issues, and other contributions that are not aligned to this Code of Conduct, or to ban temporarily or permanently any contributor for other behaviors that they deem inappropriate, threatening, offensive, or harmful.
"},{"location":"contributing/index.html#scope","title":"Scope","text":"This Code of Conduct applies both within project spaces and in public spaces when an individual is representing the project or its community. Examples of representing a project or community include using an official project e-mail address, posting via an official social media account, or acting as an appointed representative at an online or offline event. Representation of a project may be further defined and clarified by project maintainers.
"},{"location":"contributing/index.html#enforcement","title":"Enforcement","text":"Instances of abusive, harassing, or otherwise unacceptable behavior may be reported by contacting the project team at opensource@aiven.io. All complaints will be reviewed and investigated and will result in a response that is deemed necessary and appropriate to the circumstances. The project team is obligated to maintain confidentiality with regard to the reporter of an incident. Further details of specific enforcement policies may be posted separately.
Project maintainers who do not follow or enforce the Code of Conduct in good faith may face temporary or permanent repercussions as determined by other members of the project's leadership.
"},{"location":"contributing/developer-guide.html","title":"Developer guide","text":""},{"location":"contributing/developer-guide.html#get-started","title":"Get Started","text":"To get started, make sure you have a working Go environment and clone the repository:
git clone git@github.com:aiven/aiven-operator.git\ncd aiven-operator\nBefore proceeding, please ensure that you have the following dependencies installed:
-w0 flag, which may cause some tests to not work properly)Create an Aiven authentication token and an Aiven project, you'll need them in the following sections:
AIVEN_TOKEN \u2014 your authentication tokenAIVEN_PROJECT_NAME \u2014 the name of you Aiven project to run the services inOptional dependencies:
The project uses the make build system. To see the available targets, run make help.
Building the operator binary:
make build\nTo run the operator locally, first create a local k8s cluster with kind:
$ kind create cluster --image kindest/node:v1.26.6 --wait 5m\nCreating cluster \"kind\" ...\n \u2713 Ensuring node image (kindest/node:v1.26.6) \ud83d\uddbc\n \u2713 Preparing nodes \ud83d\udce6\n \u2713 Writing configuration \ud83d\udcdc\n \u2713 Starting control-plane \ud83d\udd79\ufe0f\n \u2713 Installing CNI \ud83d\udd0c\n \u2713 Installing StorageClass \ud83d\udcbe\n \u2713 Waiting \u2264 5m0s for control-plane = Ready \u23f3\n \u2022 Ready after 14s \ud83d\udc9a\nSet kubectl context to \"kind-kind\"\nYou can now use your cluster with:\n\nkubectl cluster-info --context kind-kinds\nThis creates a k8s cluster and points your kubectl to that cluster:
$ kubectl cluster-info --context kind-kind\nKubernetes control plane is running at https://127.0.0.1:58083\nCoreDNS is running at https://127.0.0.1:58083/api/v1/namespaces/kube-system/services/kube-dns:dns/proxy\nRun the operator:
ENABLE_WEBHOOKS=false make run\nIn another terminal window, create a Kubernetes Secret named aiven-token (use the Aiven authentication token created earlier):
export AIVEN_TOKEN=...\nkubectl create secret generic aiven-token --from-literal=token=$AIVEN_TOKEN\nThen, create a manifest for an Aiven PostgreSQL service:
aiven-pg.yamlapiVersion: aiven.io/v1alpha1\nkind: PostgreSQL\nmetadata:\n name: aiven-devdocs-pg\nspec:\n # reads the authentication token\n authSecretRef:\n name: aiven-token\n key: token\n\n # stores the PostgreSQL connection information on the specified Secret\n connInfoSecretTarget:\n name: pg-connection\n\n project: <your-project-name>\n cloudName: google-europe-west1\n plan: hobbyist\n maintenanceWindowDow: friday\n maintenanceWindowTime: 23:00:00\n userConfig:\n pg_version: \"15\"\n...and create the service at Aiven by applying the manifest:
kubectl apply -f aiven-pg.yaml\nYou should see the service be created on Aiven's side using either Aiven Console or Aiven CLI:
$ avn project switch PROJECT_NAME\n$ avn service list\nSERVICE_NAME SERVICE_TYPE STATE CLOUD_NAME PLAN CREATE_TIME UPDATE_TIME NOTIFICATIONS\n================ ============ ========== =================== ======== ==================== ==================== =============\naiven-devdocs-pg pg REBUILDING google-europe-west1 hobbyist 2024-01-01T00:00:00Z 2024-01-01T00:01:00Z\nTo see the resource on k8s side, use kubectl:
$ kubectl get postgresqls\nNAME PROJECT REGION PLAN STATE\naiven-devdocs-pg aiven-sbox-k8s-riski google-europe-west1 hobbyist REBUILDING\n... or by running k9s in another terminal window and type \": postgresqls\" to see the provisioned PostgreSQL resources:
For this section you'll need to set export AIVEN_TOKEN=... and an export AIVEN_PROJECT_NAME=... as well as export ENABLE_WEBHOOKS=false (see Prerequisites).
Note
If you encounter any issues with webhooks, you have the option to disable them by running export ENABLE_WEBOOKS=false
Running all unit tests is possible by running make test. To run a targeted test, use run=TestGrafana make test.
Setup the test environment:
make e2e-setup-kind\nThe e2e tests are configured to run against a project called aiven-ci-kubernetes-operator. To run the test against
make test-e2e-preinstalled\nTo remove the kind cluster, simply execute the following command:
kind delete cluster\nPlease see this page for more information.
"},{"location":"contributing/developer-guide.html#documentation","title":"Documentation","text":"The documentation is written in markdown and generated by mkdocs and mkdocs-material.
To run the documentation live preview:
make serve-docs\nAnd open the http://localhost:8000/aiven-operator/ page in your web browser.
The documentation API Reference section is generated automatically from the source code during the documentation deployment. To generate it locally, run the following command:
make docs\nAiven Kubernetes Operator generates service configs code (also known as user configs) and documentation from public service types schema.
"},{"location":"contributing/resource-generation.html#flow-overview","title":"Flow overview","text":"When a new schema is issued on the API, a cron job fetches it, parses, patches, and saves in a shared library \u2014 go-api-schemas.
When the library is updated, the GitHub dependabot creates PRs to the dependent repositories, like Aiven Kubernetes Operator and Aiven Terraform Provider.
Then the make generate command is called by GitHub action. And the PR is ready for review.
flowchart TB\n API(Aiven API) <-.->|polls schema updates| Schema([go-api-schemas])\n Bot(dependabot) <-.->|polls updates| Schema\n Bot-->|pull request|UpdateOP[/\"\u2728 $ make generate \u2728\"/]\n UpdateOP-->|review| OP([operator repository])The command runs several generators in a certain sequence. First, the user config generator is called. Then controller-gen cli. Then API reference docs generator and charts generator.
Here how it goes in the details:
./<api-reference-docs>/example/, if it finds one, it validates that with the CRD. Each CRD has an OpenAPI v3 schema as a part of it. This is also used by Kubernetes itself to validate user input.flowchart TB\n Make[/$ make generate/]-->Generator(userconfig generator<br> creates/updates structs using updated spec)\n Generator-->|go: KafkaUserConfig struct| K8S(controller-gen<br> adds k8s methods to structs)\n K8S-->|go files| CRD(controller-gen<br> creates CRDs out of structs)\n CRD-->|CRD: aiven.io_kafkas.yaml| Docs(docs generator)\n subgraph API reference generation\n Docs-->|aiven.io_kafkas.yaml|Reference(creates reference<br> out of CRD)\n Docs-->|examples/kafka.yaml,<br> aiven.io_kafkas.yaml|Examples(validates example<br> using CRD)\n Examples--> Markdown(creates docs out of CRDs, adds examples)\n Reference-->Markdown(kafka.md)\n end\n CRD-->|yaml files|Charts(charts generator<br> updates helm charts<br> and the changelog)\n Charts-->ToRelease(\"Ready to release \ud83c\udf89\")\n Markdown-->ToReleaseBy default, charts generator keeps the current helm chart's version, because it doesn't know semver. You need it to do manually.
To do so run the following command with the version of your choice:
make version=v1.0.0 charts\nThe Aiven Operator for Kubernetes can be installed via Helm.
Before you start, make sure you have the prerequisites.
First add the Aiven Helm repository:
helm repo add aiven https://aiven.github.io/aiven-charts && helm repo update\nhelm install aiven-operator-crds aiven/aiven-operator-crds\nVerify the installation:
kubectl api-resources --api-group=aiven.io\nThe output is similar to the following:
NAME SHORTNAMES APIVERSION NAMESPACED KIND\nconnectionpools aiven.io/v1alpha1 true ConnectionPool\ndatabases aiven.io/v1alpha1 true Database\n... < several omitted lines >\nhelm install aiven-operator aiven/aiven-operator\nNote
Installation will fail if webhooks are enabled and the CRDs for the cert-manager are not installed.
Verify the installation:
helm status aiven-operator\nThe output is similar to the following:
NAME: aiven-operator\nLAST DEPLOYED: Fri Sep 10 15:23:26 2021\nNAMESPACE: default\nSTATUS: deployed\nREVISION: 1\nTEST SUITE: None\nIt is also possible to install the operator without webhooks enabled:
helm install aiven-operator aiven/aiven-operator --set webhooks.enabled=false\nPlease refer to the values.yaml of the chart.
"},{"location":"installation/helm.html#install-without-full-cluster-administrator-access","title":"Install without full cluster administrator access","text":"If the person installing the Helm chart does not have the necessary permissions to create cluster-wide resources such as ClusterRole and ClusterRoleBinding, a cluster administrator can manually install these roles. This ensures that the operator can function properly.
Important
Please see this page for more information.
Find out the name of your deployment:
helm list\nThe output has the name of each deployment similar to the following:
NAME NAMESPACE REVISION UPDATED STATUS CHART APP VERSION\naiven-operator default 1 2021-09-09 10:56:14.623700249 +0200 CEST deployed aiven-operator-v0.1.0 v0.1.0\naiven-operator-crds default 1 2021-09-09 10:56:05.736411868 +0200 CEST deployed aiven-operator-crds-v0.1.0 v0.1.0\nRemove the CRDs:
helm uninstall aiven-operator-crds\nThe confirmation message is similar to the following:
release \"aiven-operator-crds\" uninstalled\nRemove the operator:
helm uninstall aiven-operator\nThe confirmation message is similar to the following:
release \"aiven-operator\" uninstalled\nBefore you start, make sure you have the prerequisites.
All Aiven Operator for Kubernetes components can be installed from one YAML file that is uploaded for every release:
kubectl apply -f https://github.com/aiven/aiven-operator/releases/latest/download/deployment.yaml\nBy default the Deployment is installed into the aiven-operator-system namespace.
Assuming you installed version vX.Y.Z of the operator it can be uninstalled via
kubectl delete -f https://github.com/aiven/aiven-operator/releases/download/vX.Y.Z/deployment.yaml\nThe Aiven Operator for Kubernetes supports all major Kubernetes distributions, both locally and in the cloud.
Make sure you have the following:
The Aiven Operator for Kubernetes uses cert-manager to configure the service reference of our webhooks.
Please follow the installation instructions on their website.
Note
This is not required in the Helm installation if you select to disable webhooks, but that is not recommended outside of playground use.
The Aiven Operator for Kubernetes uses webhooks for setting defaults and enforcing invariants that are expected by the aiven API and will lead to errors if ignored.
In the future webhooks will also be used for conversion and supporting multiple CRD versions.
"},{"location":"installation/uninstalling.html","title":"Uninstall","text":"Danger
Uninstalling the Aiven Operator for Kubernetes can remove the resources created in Aiven, possibly resulting in data loss.
Depending on your installation, please follow one of:
Aiven resources need to have an accompanying secret that contains the token that is used to authorize the manipulation of that resource. If that token expired then you will not be able to delete the custom resource and deletion will also hang until the situation is resolved. The recommended approach to deal with that situation is to patch a valid token into the secret again so that proper cleanup of aiven resources can take place.
"},{"location":"installation/uninstalling.html#hanging-deletions","title":"Hanging deletions","text":"To protect the Secrets that the operator is using from deletion, it adds the finalizer finalizers.aiven.io/needed-to-delete-services to the Secret. This solves a race condition that happens when deleting a namespace, where there is a possibility of the Secret getting deleted before the resource that uses it. When the controller is deleted it may not cleanup the finalizers from all Secrets. If there is a Secret with this finalizer blocking deletion of a namespace, you can remove the finalizer by running:
kubectl patch secret <offending-secret> -p '{\"metadata\":{\"finalizers\":null}}' --type=merge\nAiven for Apache Cassandra\u00ae is a distributed database designed to handle large volumes of writes.
Note
Before going through this guide, make sure you have a Kubernetes cluster with the operator installed (see instructions for helm or kubectl), and a Kubernetes Secret with an Aiven authentication token.
"},{"location":"resources/cassandra.html#create-a-cassandra-instance","title":"Create a Cassandra instance","text":"1. Create a file named cassandra-sample.yaml, and add the following content:
apiVersion: aiven.io/v1alpha1\nkind: Cassandra\nmetadata:\n name: cassandra-sample\nspec:\n # gets the authentication token from the `aiven-token` Secret\n authSecretRef:\n name: aiven-token\n key: token\n\n # outputs the Cassandra connection on the `cassandra-secret` Secret\n connInfoSecretTarget:\n name: cassandra-secret\n\n # add your Project name here\n project: PROJECT_NAME\n\n # cloud provider and plan of your choice\n # you can check all of the possibilities here https://aiven.io/pricing\n cloudName: google-europe-west1\n plan: startup-4\n\n # general Aiven configuration\n maintenanceWindowDow: friday\n maintenanceWindowTime: 23:00:00\n2. Create the service by applying the configuration:
kubectl apply -f cassandra-sample.yaml\nThe output is:
cassandra.aiven.io/cassandra-sample created\n3. Review the resource you created with this command:
kubectl describe cassandra.aiven.io cassandra-sample\nThe output is similar to the following:
...\nStatus:\n Conditions:\n Last Transition Time: 2023-01-31T10:17:25Z\n Message: Successfully created or updated the instance in Aiven\n Reason: Created\n Status: True\n Type: Initialized\n Last Transition Time: 2023-01-31T10:24:00Z\n Message: Instance is running on Aiven side\n Reason: CheckRunning\n Status: True\n Type: Running\n State: RUNNING\n...\nThe resource can be in the REBUILDING state for a few minutes. Once the state changes to RUNNING, you can access the resource.
For your convenience, the operator automatically stores the Cassandra connection information in a Secret created with the name specified on the connInfoSecretTarget field.
To view the details of the Secret, use the following command:
kubectl describe secret cassandra-secret\nThe output is similar to the following:
Name: cassandra-secret\nNamespace: default\nLabels: <none>\nAnnotations: <none>\n\nType: Opaque\n\nData\n====\nCASSANDRA_HOSTS: 59 bytes\nCASSANDRA_PASSWORD: 24 bytes\nCASSANDRA_PORT: 5 bytes\nCASSANDRA_URI: 66 bytes\nCASSANDRA_USER: 8 bytes\nCASSANDRA_HOST: 60 bytes\nYou can use the jq to quickly decode the Secret:
kubectl get secret cassandra-secret -o json | jq '.data | map_values(@base64d)'\nThe output is similar to the following:
{\n \"CASSANDRA_HOST\": \"<secret>\",\n \"CASSANDRA_HOSTS\": \"<secret>\",\n \"CASSANDRA_PASSWORD\": \"<secret>\",\n \"CASSANDRA_PORT\": \"14609\",\n \"CASSANDRA_URI\": \"<secret>\",\n \"CASSANDRA_USER\": \"avnadmin\"\n}\nYou can create service users for your instance of Aiven for Apache Cassandra. Service users are unique to this instance and are not shared with any other services.
1. Create a file named cassandra-service-user.yaml:
apiVersion: aiven.io/v1alpha1\nkind: ServiceUser\nmetadata:\n name: cassandra-service-user\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n connInfoSecretTarget:\n name: cassandra-service-user-secret\n\n project: PROJECT_NAME\n serviceName: cassandra-sample\n2. Create the user by applying the configuration:
kubectl apply -f cassandra-service-user.yaml\nThe ServiceUser resource generates a Secret with connection information.
3. View the details of the Secret using the following command:
kubectl get secret cassandra-service-user-secret -o json | jq '.data | map_values(@base64d)'\nThe output is similar to the following:
{\n \"ACCESS_CERT\": \"<secret>\",\n \"ACCESS_KEY\": \"<secret>\",\n \"CA_CERT\": \"<secret>\",\n \"HOST\": \"<secret>\",\n \"PASSWORD\": \"<secret>\",\n \"PORT\": \"14609\",\n \"USERNAME\": \"cassandra-service-user\"\n}\nYou can connect to the Cassandra instance using these credentials and the host information from the cassandra-secret Secret.
Aiven for ClickHouse\u00ae is a fully managed distributed columnar database based on open source ClickHouse.
Note
Before going through this guide, make sure you have a Kubernetes cluster with the operator installed (see instructions for helm or kubectl), and a Kubernetes Secret with an Aiven authentication token.
"},{"location":"resources/clickhouse.html#create-a-clickhouse-instance","title":"Create a ClickHouse instance","text":"1. Create a file named clickhouse-sample.yaml, and add the following:
apiVersion: aiven.io/v1alpha1\nkind: Clickhouse\nmetadata:\n name: example-clickhouse\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n # Outputs the ClickHouse connection to the `clickhouse-secret` Secret.\n connInfoSecretTarget:\n name: clickhouse-secret\n # Your Aiven project.\n project: PROJECT_NAME\n # Choose a cloud provider and plan.\n # View the options on the pricing page at https://aiven.io/pricing.\n cloudName: google-europe-west1\n plan: startup-16\n # Configure the maintenance window.\n maintenanceWindowDow: friday\n maintenanceWindowTime: 23:00:00\nPROJECT_NAME is the name of your Aiven project. 2. To create the service, run:
kubectl apply -f clickhouse-sample.yaml\nThe output is similar to the following:
clickhouse.aiven.io/example-clickhouse created\n3. To review the resource you created, run:
kubectl describe clickhouse.aiven.io example-clickhouse\nThe output is similar to the following:
...\nStatus:\n Conditions:\n Last Transition Time: 2024-06-25T07:58:37Z\n Message: Instance was created or update on Aiven side\n Reason: Created\n Status: True\n Type: Initialized\n Last Transition Time: 2024-06-25T08:01:47Z\n Message: Instance is running on Aiven side\n Reason: CheckRunning\n Status: True\n Type: Running\n State: RUNNING\nEvents:\n Type Reason Age From Message\n ---- ------ ---- ---- -------\n Normal CreatedOrUpdatedAtAiven 3m24s (x5 over 3m45s) clickhouse-reconciler waiting for the instance to be running\n Normal ReconcilationStarted 3m14s (x6 over 3m47s) clickhouse-reconciler starting reconciliation\n Normal InstanceFinalizerAdded 3m14s (x6 over 3m47s) clickhouse-reconciler waiting for preconditions of the instance\n...\nREBUILDING state for a few minutes. Once the state changes to RUNNING, you can access the service."},{"location":"resources/clickhouse.html#use-the-connection-secret","title":"Use the connection Secret","text":"Aiven Operator automatically stores the ClickHouse connection information in a Secret created with the name in the connInfoSecretTarget field.
To view the details of the Secret, run:
kubectl describe secret clickhouse-secret\nName: clickhouse-secret\nNamespace: default\nLabels: <none>\nAnnotations: <none>\n\nType: Opaque\n\nData\n====\nCLICKHOUSE_USER: 8 bytes\nHOST: 46 bytes\nPASSWORD: 24 bytes\nPORT: 5 bytes\nUSER: 8 bytes\nCLICKHOUSE_HOST: 46 bytes\nCLICKHOUSE_PASSWORD: 24 bytes\nCLICKHOUSE_PORT: 5 bytes\nYou can use the JSON processor jq to decode the Secret:
kubectl get secret clickhouse-secret -o json | jq '.data | map_values(@base64d)'\nThe output is similar to the following:
{\n \"CLICKHOUSE_HOST\": \"HOST\",\n \"CLICKHOUSE_PASSWORD\": \"SERVICE_PASSWORD\",\n \"CLICKHOUSE_PORT\": \"12691\",\n \"CLICKHOUSE_USER\": \"avnadmin\",\n \"HOST\": \"HOST\",\n \"PASSWORD\": \"ADMIN_USER_PASSWORD\",\n \"PORT\": \"12691\",\n \"USER\": \"avnadmin\"\n}\nNote
Tables cannot be created using Aiven Operator. To create a table, use the Aiven Console or CLI.
1. Create a file named clickhouse-db.yaml and add the following:
apiVersion: aiven.io/v1alpha1\nkind: ClickhouseDatabase\nmetadata:\n name: example-database\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n serviceName: example-clickhouse\n project: PROJECT_NAME\nWhere PROJECT_NAME is the name of your Aiven project.
2. To create the database, run:
kubectl apply -f clickhouse-db.yaml\n3. To view details about the resource, run:
kubectl describe ClickhouseDatabase example-database\nThe output is similar to the following:
...\nSpec:\n Auth Secret Ref:\n Key: token\n Name: aiven-token\n Project: example-project\n Service Name: example-clickhouse\nStatus:\n Conditions:\n Last Transition Time: 2024-06-25T13:58:35Z\n Message: Checking preconditions\n Reason: Preconditions\n Status: True\n Type: Initialized\n Last Transition Time: 2024-06-25T14:32:05Z\n Message: Instance is running on Aiven side\n Reason: CheckRunning\n Status: True\n Type: Running\n...\nYou can create service users and roles for an instance of ClickHouse, and grant privileges to them. Users and roles are not shared with any other services.
1. Create a file named clickhouse-service-users.yaml and add the following configuration for a service user:
apiVersion: aiven.io/v1alpha1\nkind: ClickhouseUser\nmetadata:\n name: example-user\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n connInfoSecretTarget:\n name: clickhouse-service-user-secret\n serviceName: example-clickhouse\n project: PROJECT_NAME\nPROJECT_NAME is the name of your Aiven project. This resource generates a Secret with connection information and stores it in clickhouse-service-user-secret.
2. To create a role add the following to the same file:
---\n\napiVersion: aiven.io/v1alpha1\nkind: ClickhouseRole\nmetadata:\n name: example-role\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n serviceName: example-clickhouse\n project: PROJECT_NAME\n role: read-only\n3. Grant privileges to the role, and assign the role to the user by adding the following to the same file:
---\n\napiVersion: aiven.io/v1alpha1\nkind: ClickhouseGrant\nmetadata:\n name: example-grant\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n project: PROJECT_NAME\n serviceName: example-clickhouse\n\n privilegeGrants:\n - grantees:\n - role: read-only\n privileges:\n - SELECT\n database: example-database\n\n roleGrants:\n - roles:\n - read-only\n grantees:\n - user: example-user\n4. To create the user, role, and grant, run:
kubectl apply -f clickhouse-service-users.yaml\n5. To get credentials and host information for connecting to this instance, view the connection information stored in the Secret by running:
kubectl get secret clickhouse-service-user-secret -o json | jq '.data | map_values(@base64d)'\nThe output is similar to the following:
{\n \"CLICKHOUSEUSER_HOST\": \"HOST\",\n \"CLICKHOUSEUSER_PASSWORD\": \"PASSWORD\",\n \"CLICKHOUSEUSER_PORT\": \"12691\",\n \"CLICKHOUSEUSER_USERNAME\": \"example-user\",\n \"HOST\": \"HOST\",\n \"PASSWORD\": \"PASSWORD\",\n \"PORT\": \"12691\",\n \"USERNAME\": \"example-user\"\n}\nAiven for MySQL is a fully managed relational database service, deployable in the cloud of your choice.
Before going through this guide, make sure you have a Kubernetes cluster with the operator installed (see instructions for helm or kubectl) and a Kubernetes Secret with an Aiven authentication token.
"},{"location":"resources/mysql.html#create-a-mysql-instance","title":"Create a MySQL instance","text":"1. Create a file named mysql-sample.yaml, and add the following content:
apiVersion: aiven.io/v1alpha1\nkind: MySQL\nmetadata:\n name: mysql-sample\nspec:\n # gets the authentication token from the `aiven-token` Secret\n authSecretRef:\n name: aiven-token\n key: token\n\n # outputs the MySQL connection on the `mysql-secret` Secret\n connInfoSecretTarget:\n name: mysql-secret\n\n # add your Project name here\n project: PROJECT_NAME\n\n # cloud provider and plan of your choice\n # you can check all of the possibilities here https://aiven.io/pricing\n cloudName: google-europe-west1\n plan: startup-4\n\n # general Aiven configuration\n maintenanceWindowDow: friday\n maintenanceWindowTime: 23:00:00\n2. Create the service by applying the configuration:
kubectl apply -f mysql-sample.yaml\n3. Review the resource you created with this command:
kubectl describe mysql.aiven.io mysql-sample\nThe output is similar to the following:
...\nStatus:\n Conditions:\n Last Transition Time: 2023-02-22T15:43:44Z\n Message: Successfully created or updated the instance in Aiven\n Reason: Created\n Status: True\n Type: Initialized\n Last Transition Time: 2023-02-22T15:43:44Z\n Message: Successfully created or updated the instance in Aiven, status remains unknown\n Reason: Created\n Status: Unknown\n Type: Running\n State: REBUILDING\n...\nThe resource will be in the REBUILDING state for a few minutes. Once the state changes to RUNNING, you can access the resource.
For your convenience, the operator automatically stores the MySQL connection information in a Secret created with the name specified on the connInfoSecretTarget field.
To view the details of the Secret, use the following command:
kubectl describe secret mysql-secret\nThe output is similar to the following:
Name: mysql-secret\nNamespace: default\nLabels: <none>\nAnnotations: <none>\n\nType: Opaque\n\nData\n====\nMYSQL_PORT: 5 bytes\nMYSQL_SSL_MODE: 8 bytes\nMYSQL_URI: 115 bytes\nMYSQL_USER: 8 bytes\nMYSQL_DATABASE: 9 bytes\nMYSQL_HOST: 39 bytes\nMYSQL_PASSWORD: 24 bytes\nYou can use jq to quickly decode the Secret:
kubectl get secret mysql-secret -o json | jq '.data | map_values(@base64d)'\nThe output is similar to the following:
{\n \"MYSQL_DATABASE\": \"defaultdb\",\n \"MYSQL_HOST\": \"<secret>\",\n \"MYSQL_PASSWORD\": \"<secret>\",\n \"MYSQL_PORT\": \"12691\",\n \"MYSQL_SSL_MODE\": \"REQUIRED\",\n \"MYSQL_URI\": \"<secret>\",\n \"MYSQL_USER\": \"avnadmin\"\n}\nYou can create service users for your instance of Aiven for MySQL. Service users are unique to this instance and are not shared with any other services.
1. Create a file named mysql-service-user.yaml:
apiVersion: aiven.io/v1alpha1\nkind: ServiceUser\nmetadata:\n name: mysql-service-user\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n connInfoSecretTarget:\n name: mysql-service-user-secret\n\n project: PROJECT_NAME\n serviceName: mysql-sample\n2. Create the user by applying the configuration:
kubectl apply -f mysql-service-user.yaml\nThe ServiceUser resource generates a Secret with connection information.
3. View the details of the Secret using jq:
kubectl get secret mysql-service-user-secret -o json | jq '.data | map_values(@base64d)'\nThe output is similar to the following:
{\n \"ACCESS_CERT\": \"<secret>\",\n \"ACCESS_KEY\": \"<secret>\",\n \"CA_CERT\": \"<secret>\",\n \"HOST\": \"<secret>\",\n \"PASSWORD\": \"<secret>\",\n \"PORT\": \"14609\",\n \"USERNAME\": \"mysql-service-user\"\n}\nYou can connect to the MySQL instance using these credentials and the host information from the mysql-secret Secret.
OpenSearch\u00ae is an open source search and analytics suite including search engine, NoSQL document database, and visualization interface. OpenSearch offers a distributed, full-text search engine based on Apache Lucene\u00ae with a RESTful API interface and support for JSON documents.
Note
Before going through this guide, make sure you have a Kubernetes cluster with the operator installed (see instructions for helm or kubectl), and a Kubernetes Secret with an Aiven authentication token.
"},{"location":"resources/opensearch.html#create-an-opensearch-instance","title":"Create an OpenSearch instance","text":"1. Create a file named os-sample.yaml, and add the following content:
apiVersion: aiven.io/v1alpha1\nkind: OpenSearch\nmetadata:\n name: os-sample\nspec:\n # gets the authentication token from the `aiven-token` Secret\n authSecretRef:\n name: aiven-token\n key: token\n\n # outputs the OpenSearch connection on the `os-secret` Secret\n connInfoSecretTarget:\n name: os-secret\n\n # add your Project name here\n project: PROJECT_NAME\n\n # cloud provider and plan of your choice\n # you can check all of the possibilities here https://aiven.io/pricing\n cloudName: google-europe-west1\n plan: startup-4\n\n # general Aiven configuration\n maintenanceWindowDow: friday\n maintenanceWindowTime: 23:00:00\n2. Create the service by applying the configuration:
kubectl apply -f os-sample.yaml\n3. Review the resource you created with this command:
kubectl describe opensearch.aiven.io os-sample\nThe output is similar to the following:
...\nStatus:\n Conditions:\n Last Transition Time: 2023-01-19T14:41:43Z\n Message: Successfully created or updated the instance in Aiven\n Reason: Created\n Status: True\n Type: Initialized\n Last Transition Time: 2023-01-19T14:41:43Z\n Message: Successfully created or updated the instance in Aiven, status remains unknown\n Reason: Created\n Status: Unknown\n Type: Running\n State: REBUILDING\n...\nThe resource will be in the REBUILDING state for a few minutes. Once the state changes to RUNNING, you can access the resource.
For your convenience, the operator automatically stores the OpenSearch connection information in a Secret created with the name specified on the connInfoSecretTarget field.
To view the details of the Secret, use the following command:
kubectl describe secret os-secret\nThe output is similar to the following:
Name: os-secret\nNamespace: default\nLabels: <none>\nAnnotations: <none>\n\nType: Opaque\n\nData\n====\nHOST: 61 bytes\nPASSWORD: 24 bytes\nPORT: 5 bytes\nUSER: 8 bytes\nYou can use the jq to quickly decode the Secret:
kubectl get secret os-secret -o json | jq '.data | map_values(@base64d)'\nThe output is similar to the following:
{\n \"HOST\": \"os-sample-your-project.aivencloud.com\",\n \"PASSWORD\": \"<secret>\",\n \"PORT\": \"13041\",\n \"USER\": \"avnadmin\"\n}\nYou can create service users for your instance of Aiven for OpenSearch. Service users are unique to this instance and are not shared with any other services.
1. Create a file named os-service-user.yaml:
apiVersion: aiven.io/v1alpha1\nkind: ServiceUser\nmetadata:\n name: os-service-user\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n connInfoSecretTarget:\n name: os-service-user-secret\n\n project: PROJECT_NAME\n serviceName: os-sample\n2. Create the user by applying the configuration:
kubectl apply -f os-service-user.yaml\nThe ServiceUser resource generates a Secret with connection information.
3. View the details of the Secret using the following command:
kubectl get secret os-service-user-secret -o json | jq '.data | map_values(@base64d)'\nThe output is similar to the following:
{\n \"ACCESS_CERT\": \"<secret>\",\n \"ACCESS_KEY\": \"<secret>\",\n \"CA_CERT\": \"<secret>\",\n \"HOST\": \"os-sample-your-project.aivencloud.com\",\n \"PASSWORD\": \"<secret>\",\n \"PORT\": \"14609\",\n \"USERNAME\": \"os-service-user\"\n}\nYou can connect to the OpenSearch instance using these credentials and the host information from the os-secret Secret.
PostgreSQL is an open source, relational database. It's ideal for organisations that need a well organised tabular datastore. On top of the strict table and columns formats, PostgreSQL also offers solutions for nested datasets with the native jsonb format and advanced set of extensions including PostGIS, a spatial database extender for location queries. Aiven for PostgreSQL is the perfect fit for your relational data.
With Aiven Kubernetes Operator, you can manage Aiven for PostgreSQL through the well defined Kubernetes API.
Note
Before going through this guide, make sure you have a Kubernetes cluster with the operator installed (see instructions for helm or kubectl), and a Kubernetes Secret with an Aiven authentication token.
"},{"location":"resources/postgresql.html#create-a-postgresql-instance","title":"Create a PostgreSQL instance","text":"1. Create a file named pg-sample.yaml with the following content:
apiVersion: aiven.io/v1alpha1\nkind: PostgreSQL\nmetadata:\n name: pg-sample\nspec:\n # gets the authentication token from the `aiven-token` Secret\n authSecretRef:\n name: aiven-token\n key: token\n\n # outputs the PostgreSQL connection on the `pg-connection` Secret\n connInfoSecretTarget:\n name: pg-connection\n\n # add your Project name here\n project: PROJECT_NAME\n\n # cloud provider and plan of your choice\n # you can check all of the possibilities here https://aiven.io/pricing\n cloudName: google-europe-west1\n plan: startup-4\n\n # general Aiven configuration\n maintenanceWindowDow: friday\n maintenanceWindowTime: 23:00:00\n\n # specific PostgreSQL configuration\n userConfig:\n pg_version: \"11\"\n2. Create the service by applying the configuration:
kubectl apply -f pg-sample.yaml\n3. Review the resource you created with the following command:
kubectl get postgresqls.aiven.io pg-sample\nThe output is similar to the following:
NAME PROJECT REGION PLAN STATE\npg-sample your-project google-europe-west1 startup-4 RUNNING\nThe resource can stay in the BUILDING state for a couple of minutes. Once the state changes to RUNNING, you are ready to access it.
For your convenience, the operator automatically stores the PostgreSQL connection information in a Secret created with the name specified on the connInfoSecretTarget field.
kubectl describe secret pg-connection\nThe output is similar to the following:
Name: pg-connection\nNamespace: default\nAnnotations: <none>\n\nType: Opaque\n\nData\n====\nDATABASE_URI: 107 bytes\nPGDATABASE: 9 bytes\nPGHOST: 38 bytes\nPGPASSWORD: 16 bytes\nPGPORT: 5 bytes\nPGSSLMODE: 7 bytes\nPGUSER: 8 bytes\nYou can use the jq to quickly decode the Secret:
kubectl get secret pg-connection -o json | jq '.data | map_values(@base64d)'\nThe output is similar to the following:
{\n \"DATABASE_URI\": \"postgres://avnadmin:<secret-password>@pg-sample-your-project.aivencloud.com:13039/defaultdb?sslmode=require\",\n \"PGDATABASE\": \"defaultdb\",\n \"PGHOST\": \"pg-sample-your-project.aivencloud.com\",\n \"PGPASSWORD\": \"<secret-password>\",\n \"PGPORT\": \"13039\",\n \"PGSSLMODE\": \"require\",\n \"PGUSER\": \"avnadmin\"\n}\nYou can verify your PostgreSQL connection from a Kubernetes workload by deploying a Pod that runs the psql command.
1. Create a file named pod-psql.yaml
apiVersion: v1\nkind: Pod\nmetadata:\n name: psql-test-connection\nspec:\n restartPolicy: Never\n containers:\n - image: postgres:11-alpine\n name: postgres\n command: [\"psql\", \"$(DATABASE_URI)\", \"-c\", \"SELECT version();\"]\n\n # the pg-connection Secret becomes environment variables\n envFrom:\n - secretRef:\n name: pg-connection\nIt runs once and stops, due to the restartPolicy: Never flag.
2. Inspect the log:
kubectl logs psql-test-connection\nThe output is similar to the following:
version\n---------------------------------------------------------------------------------------------\n PostgreSQL 11.12 on x86_64-pc-linux-gnu, compiled by gcc, a 68c5366192 p 6b9244f01a, 64-bit\n(1 row)\nYou have now connected to the PostgreSQL, and executed the SELECT version(); query.
The Database Kubernetes resource allows you to create a logical database within the PostgreSQL instance.
Create the pg-database-sample.yaml file with the following content:
apiVersion: aiven.io/v1alpha1\nkind: Database\nmetadata:\n name: pg-database-sample\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n # the name of the previously created PostgreSQL instance\n serviceName: pg-sample\n\n project: PROJECT_NAME\n lcCollate: en_US.UTF-8\n lcCtype: en_US.UTF-8\nYou can now connect to the pg-database-sample using the credentials stored in the pg-connection Secret.
Aiven uses the concept of service user that allows you to create users for different services. You can create one for the PostgreSQL instance.
1. Create a file named pg-service-user.yaml.
apiVersion: aiven.io/v1alpha1\nkind: ServiceUser\nmetadata:\n name: pg-service-user\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n connInfoSecretTarget:\n name: pg-service-user-connection\n\n project: PROJECT_NAME\n serviceName: pg-sample\n2. Apply the configuration with the following command.
kubectl apply -f pg-service-user.yaml\nThe ServiceUser resource generates a Secret with connection information, in this case named pg-service-user-connection:
kubectl get secret pg-service-user-connection -o json | jq '.data | map_values(@base64d)'\nThe output has the password and username:
{\n \"PASSWORD\": \"<secret-password>\",\n \"USERNAME\": \"pg-service-user\"\n}\nYou can now connect to the PostgreSQL instance using the credentials generated above, and the host information from the pg-connection Secret.
Connection pooling allows you to maintain very large numbers of connections to a database while minimizing the consumption of server resources. For more information, refer to the connection pooling article in Aiven Docs. Aiven for PostgreSQL uses PGBouncer for connection pooling.
You can create a connection pool with the ConnectionPool resource using the previously created Database and ServiceUser:
Create a new file named pg-connection-pool.yaml with the following content:
apiVersion: aiven.io/v1alpha1\nkind: ConnectionPool\nmetadata:\n name: pg-connection-pool\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n connInfoSecretTarget:\n name: pg-connection-pool-connection\n\n project: PROJECT_NAME\n serviceName: pg-sample\n databaseName: pg-database-sample\n username: pg-service-user\n poolSize: 10\n poolMode: transaction\nThe ConnectionPool generates a Secret with the connection info using the name from the connInfoSecretTarget.Name field:
kubectl get secret pg-connection-pool-connection -o json | jq '.data | map_values(@base64d)'\nThe output is similar to the following:
{\n \"DATABASE_URI\": \"postgres://pg-service-user:<secret-password>@pg-sample-you-project.aivencloud.com:13040/pg-connection-pool?sslmode=require\",\n \"PGDATABASE\": \"pg-database-sample\",\n \"PGHOST\": \"pg-sample-your-project.aivencloud.com\",\n \"PGPASSWORD\": \"<secret-password>\",\n \"PGPORT\": \"13040\",\n \"PGSSLMODE\": \"require\",\n \"PGUSER\": \"pg-service-user\"\n}\nRead-only replicas can be used to reduce the load on the primary service by making read-only queries against the replica service.
To create a read-only replica for a PostgreSQL service, you create a second PostgreSQL service and use serviceIntegrations to replicate data from your primary service.
The example that follows creates a primary service and a read-only replica.
1. Create a new file named pg-read-replica.yaml with the following:
apiVersion: aiven.io/v1alpha1\nkind: PostgreSQL\nmetadata:\n name: primary-pg-service\nspec:\n # gets the authentication token from the `aiven-token` Secret\n authSecretRef:\n name: aiven-token\n key: token\n\n # add your project's name here\n project: PROJECT_NAME\n\n # add the cloud provider and plan of your choice\n # you can see all of the options at https://aiven.io/pricing\n cloudName: google-europe-west1\n plan: startup-4\n\n # general Aiven configuration\n maintenanceWindowDow: friday\n maintenanceWindowTime: 23:00:00\n userConfig:\n pg_version: \"15\"\n\n---\napiVersion: aiven.io/v1alpha1\nkind: PostgreSQL\nmetadata:\n name: read-replica-pg\nspec:\n # gets the authentication token from the `aiven-token` Secret\n authSecretRef:\n name: aiven-token\n key: token\n\n # add your project's name here\n project: PROJECT_NAME\n\n # add the cloud provider and plan of your choice\n # you can see all of the options at https://aiven.io/pricing\n cloudName: google-europe-west1\n plan: startup-4\n\n # general Aiven configuration\n maintenanceWindowDow: saturday\n maintenanceWindowTime: 23:00:00\n userConfig:\n pg_version: \"15\"\n\n # use the read_replica integration and point it to your primary service\n serviceIntegrations:\n - integrationType: read_replica\n sourceServiceName: primary-pg-service\nNote
You can create the replica service in a different region or on a different cloud provider.
2. Apply the configuration with the following command:
kubectl apply -f pg-read-replica.yaml\nThe output is similar to the following:
postgresql.aiven.io/primary-pg-service created\npostgresql.aiven.io/read-replica-pg created\n3. Check the status of the primary service with the following command:
kubectl get postgresqls.aiven.io primary-pg-service\nThe output is similar to the following:
NAME PROJECT REGION PLAN STATE\nprimary-pg-service PROJECT_NAME google-europe-west1 startup-4 RUNNING\nThe resource can be in the BUILDING state for a few minutes. After the state of the primary service changes to RUNNING, the read-only replica is created. You can check the status of the replica using the same command with the name of the replica:
kubectl get postgresqls.aiven.io read-replica-pg\nVirtual Private Cloud (VPC) peering is a method of connecting separate AWS, Google Cloud or Microsoft Azure private networks to each other. It makes it possible for the virtual machines in the different VPCs to talk to each other directly without going through the public internet.
Within the Aiven Kubernetes Operator, you can create a ProjectVPC on Aiven's side to connect to your cloud provider.
Note
Before going through this guide, make sure you have a Kubernetes cluster with the operator installed (see instructions for helm or kubectl), and a Kubernetes Secret with an Aiven authentication token.
"},{"location":"resources/project-vpc.html#create-an-aiven-vpc","title":"Create an Aiven VPC","text":"1. Create a file named vpc-sample.yaml with the following content:
apiVersion: aiven.io/v1alpha1\nkind: ProjectVPC\nmetadata:\n name: vpc-sample\nspec:\n # gets the authentication token from the `aiven-token` Secret\n authSecretRef:\n name: aiven-token\n key: token\n\n project: PROJECT_NAME\n\n # creates a VPC to link an AWS account on the South Africa region\n cloudName: aws-af-south-1\n\n # the network range used by the VPC\n networkCidr: 192.168.0.0/24\n2. Create the Project by applying the configuration:
kubectl apply -f vpc-sample.yaml\n3. Review the resource you created with the following command:
kubectl get projects.aiven.io vpc-sample\nThe output is similar to the following:
NAME PROJECT CLOUD NETWORK CIDR\nvpc-sample PROJECT_NAME aws-af-south-1 192.168.0.0/24\nFollow the official VPC documentation to complete the VPC peering on your cloud of choice.
"},{"location":"resources/project.html","title":"Aiven Project","text":"Note
Before going through this guide, make sure you have a Kubernetes cluster with the operator installed (see instructions for helm or kubectl), and a Kubernetes Secret with an Aiven authentication token.
The Project CRD allows you to create Aiven Projects, where your resources can be located.
To create a fully working Aiven Project with the Aiven Operator you need a source Aiven Project already created with a working billing configuration, like a credit card.
Create a file named project-sample.yaml with the following content:
apiVersion: aiven.io/v1alpha1\nkind: Project\nmetadata:\n name: project-sample\nspec:\n # the source Project to copy the billing information from\n copyFromProject: SOURCE_PROJECT\n\n authSecretRef:\n name: aiven-token\n key: token\n\n connInfoSecretTarget:\n name: project-sample\nApply the resource with:
kubectl apply -f project-sample.yaml\nVerify the newly created Project:
kubectl get projects.aiven.io project-sample\nThe output is similar to the following:
NAME AGE\nproject-sample 22s\nAiven for Redis\u00ae* is a fully managed in-memory NoSQL database that you can deploy in the cloud of your choice to store and access data quickly and efficiently.
Note
Before going through this guide, make sure you have a Kubernetes cluster with the operator installed (see instructions for helm or kubectl), and a Kubernetes Secret with an Aiven authentication token.
"},{"location":"resources/redis.html#create-a-redis-instance","title":"Create a Redis instance","text":"1. Create a file named redis-sample.yaml, and add the following content:
apiVersion: aiven.io/v1alpha1\nkind: Redis\nmetadata:\n name: redis-sample\nspec:\n # gets the authentication token from the `aiven-token` Secret\n authSecretRef:\n name: aiven-token\n key: token\n\n # outputs the Redis connection on the `redis-secret` Secret\n connInfoSecretTarget:\n name: redis-secret\n\n # add your Project name here\n project: PROJECT_NAME\n\n # cloud provider and plan of your choice\n # you can check all of the possibilities here https://aiven.io/pricing\n cloudName: google-europe-west1\n plan: startup-4\n\n # general Aiven configuration\n maintenanceWindowDow: friday\n maintenanceWindowTime: 23:00:00\n\n # specific Redis configuration\n userConfig:\n redis_maxmemory_policy: \"allkeys-random\"\n2. Create the service by applying the configuration:
kubectl apply -f redis-sample.yaml\n3. Review the resource you created with this command:
kubectl describe redis.aiven.io redis-sample\nThe output is similar to the following:
...\nStatus:\n Conditions:\n Last Transition Time: 2023-01-19T14:48:59Z\n Message: Successfully created or updated the instance in Aiven\n Reason: Created\n Status: True\n Type: Initialized\n Last Transition Time: 2023-01-19T14:48:59Z\n Message: Successfully created or updated the instance in Aiven, status remains unknown\n Reason: Created\n Status: Unknown\n Type: Running\n State: REBUILDING\n...\nThe resource will be in the REBUILDING state for a few minutes. Once the state changes to RUNNING, you can access the resource.
For your convenience, the operator automatically stores the Redis connection information in a Secret created with the name specified on the connInfoSecretTarget field.
To view the details of the Secret, use the following command:
kubectl describe secret redis-secret\nThe output is similar to the following:
Name: redis-secret\nNamespace: default\nLabels: <none>\nAnnotations: <none>\n\nType: Opaque\n\nData\n====\nSSL: 8 bytes\nUSER: 7 bytes\nHOST: 60 bytes\nPASSWORD: 24 bytes\nPORT: 5 bytes\nYou can use the jq to quickly decode the Secret:
kubectl get secret redis-secret -o json | jq '.data | map_values(@base64d)'\nThe output is similar to the following:
{\n \"HOST\": \"redis-sample-your-project.aivencloud.com\",\n \"PASSWORD\": \"<secret-password>\",\n \"PORT\": \"14610\",\n \"SSL\": \"required\",\n \"USER\": \"default\"\n}\nService Integrations provide additional functionality and features by connecting different Aiven services together.
See our Getting Started with Service Integrations guide for more information.
Note
Before going through this guide, make sure you have a Kubernetes cluster with the operator installed (see instructions for helm or kubectl), and a Kubernetes Secret with an Aiven authentication token.
"},{"location":"resources/service-integrations.html#send-kafka-logs-to-a-kafka-topic","title":"Send Kafka logs to a Kafka Topic","text":"This integration allows you to send Kafka service logs to a specific Kafka Topic.
First, let's create a Kafka service and a topic.
1. Create a new file named kafka-sample-topic.yaml with the following content:
apiVersion: aiven.io/v1alpha1\nkind: Kafka\nmetadata:\n name: kafka-sample\nspec:\n # gets the authentication token from the `aiven-token` Secret\n authSecretRef:\n name: aiven-token\n key: token\n\n # outputs the Kafka connection on the `kafka-connection` Secret\n connInfoSecretTarget:\n name: kafka-auth\n\n # add your Project name here\n project: PROJECT_NAME\n\n # cloud provider and plan of your choice\n # you can check all of the possibilities here https://aiven.io/pricing\n cloudName: google-europe-west1\n plan: startup-2\n\n # general Aiven configuration\n maintenanceWindowDow: friday\n maintenanceWindowTime: 23:00:00\n\n # specific Kafka configuration\n userConfig:\n kafka_version: \"2.7\"\n\n---\napiVersion: aiven.io/v1alpha1\nkind: KafkaTopic\nmetadata:\n name: logs\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n project: PROJECT_NAME\n serviceName: kafka-sample\n\n # here we can specify how many partitions the topic should have\n partitions: 3\n # and the topic replication factor\n replication: 2\n\n # we also support various topic-specific configurations\n config:\n flush_ms: 100\n2. Create the resource on Kubernetes:
kubectl apply -f kafka-sample-topic.yaml\n3. Now, create a ServiceIntegration resource to send the Kafka logs to the created topic. In the same file, add the following YAML:
apiVersion: aiven.io/v1alpha1\nkind: ServiceIntegration\nmetadata:\n name: service-integration-kafka-logs\nspec:\n # gets the authentication token from the `aiven-token` Secret\n authSecretRef:\n name: aiven-token\n key: token\n\n project: PROJECT_NAME\n\n # indicates the type of the integration\n integrationType: kafka_logs\n\n # we will send the logs to the same kafka-sample instance\n # the source and destination are the same\n sourceServiceName: kafka-sample\n destinationServiceName: kafka-sample\n\n # the topic name we will send to\n kafkaLogs:\n kafka_topic: logs\n4. Reapply the resource on Kubernetes:
kubectl apply -f kafka-sample-topic.yaml\n5. Let's check the created service integration:
kubectl get serviceintegrations.aiven.io service-integration-kafka-logs\nThe output is similar to the following:
NAME PROJECT TYPE SOURCE SERVICE NAME DESTINATION SERVICE NAME SOURCE ENDPOINT ID DESTINATION ENDPOINT ID\nservice-integration-kafka-logs your-project kafka_logs kafka-sample kafka-sample\nYour Kafka service logs are now being streamed to the logs Kafka topic.
Aiven for Apache Kafka is an excellent option if you need to run Apache Kafka at scale. With Aiven Kubernetes Operator you can get up and running with a suitably sized Apache Kafka service in a few minutes.
Note
Before going through this guide, make sure you have a Kubernetes cluster with the operator installed (see instructions for helm or kubectl), and a Kubernetes Secret with an Aiven authentication token.
"},{"location":"resources/kafka/index.html#create-a-kafka-instance","title":"Create a Kafka instance","text":"1. Create a file named kafka-sample.yaml, and add the following content:
apiVersion: aiven.io/v1alpha1\nkind: Kafka\nmetadata:\n name: kafka-sample\nspec:\n # gets the authentication token from the `aiven-token` Secret\n authSecretRef:\n name: aiven-token\n key: token\n\n # outputs the Kafka connection on the `kafka-connection` Secret\n connInfoSecretTarget:\n name: kafka-auth\n\n # add your Project name here\n project: PROJECT_NAME\n\n # cloud provider and plan of your choice\n # you can check all of the possibilities here https://aiven.io/pricing\n cloudName: google-europe-west1\n plan: startup-2\n\n # general Aiven configuration\n maintenanceWindowDow: friday\n maintenanceWindowTime: 23:00:00\n\n # specific Kafka configuration\n userConfig:\n kafka_version: \"2.7\"\n2. Create the following resource on Kubernetes:
kubectl apply -f kafka-sample.yaml\n3. Inspect the service created using the command below.
kubectl get kafka.aiven.io kafka-sample\nThe output has the project name and state, similar to the following:
NAME PROJECT REGION PLAN STATE\nkafka-sample PROJECT_NAME google-europe-west1 startup-2 RUNNING\nAfter a couple of minutes, the STATE field is changed to RUNNING, and is ready to be used.
For your convenience, the operator automatically stores the Kafka connection information in a Secret created with the name specified on the connInfoSecretTarget field.
kubectl describe secret kafka-auth\nThe output is similar to the following:
Name: kafka-auth\nNamespace: default\nAnnotations: <none>\n\nType: Opaque\n\nData\n====\nCA_CERT: 1537 bytes\nHOST: 41 bytes\nPASSWORD: 16 bytes\nPORT: 5 bytes\nUSERNAME: 8 bytes\nACCESS_CERT: 1533 bytes\nACCESS_KEY: 2484 bytes\nYou can use the jq to quickly decode the Secret:
kubectl get secret kafka-auth -o json | jq '.data | map_values(@base64d)'\nThe output is similar to the following:
{\n \"CA_CERT\": \"<secret-ca-cert>\",\n \"ACCESS_CERT\": \"<secret-cert>\",\n \"ACCESS_KEY\": \"<secret-access-key>\",\n \"HOST\": \"kafka-sample-your-project.aivencloud.com\",\n \"PASSWORD\": \"<secret-password>\",\n \"PORT\": \"13041\",\n \"USERNAME\": \"avnadmin\"\n}\nYou can verify your access to the Kafka cluster from a Pod using the authentication data from the kafka-auth Secret. kcat is used for our examples below.
1. Create a file named kafka-test-connection.yaml, and add the following content:
apiVersion: v1\nkind: Pod\nmetadata:\n name: kafka-test-connection\nspec:\n restartPolicy: Never\n containers:\n - image: edenhill/kcat:1.7.0\n name: kcat\n\n # the command below will connect to the Kafka cluster\n # and output its metadata\n command:\n [\n \"kcat\",\n \"-b\",\n \"$(HOST):$(PORT)\",\n \"-X\",\n \"security.protocol=SSL\",\n \"-X\",\n \"ssl.key.location=/kafka-auth/ACCESS_KEY\",\n \"-X\",\n \"ssl.key.password=$(PASSWORD)\",\n \"-X\",\n \"ssl.certificate.location=/kafka-auth/ACCESS_CERT\",\n \"-X\",\n \"ssl.ca.location=/kafka-auth/CA_CERT\",\n \"-L\",\n ]\n\n # loading the data from the Secret as environment variables\n # useful to access the Kafka information, like hostname and port\n envFrom:\n - secretRef:\n name: kafka-auth\n\n volumeMounts:\n - name: kafka-auth\n mountPath: \"/kafka-auth\"\n\n # loading the data from the Secret as files in a volume\n # useful to access the Kafka certificates\n volumes:\n - name: kafka-auth\n secret:\n secretName: kafka-auth\n2. Apply the file.
kubectl apply -f kafka-test-connection.yaml\nOnce successfully applied, you have a log with the metadata information about the Kafka cluster.
kubectl logs kafka-test-connection\nThe output is similar to the following:
Metadata for all topics (from broker -1: ssl://kafka-sample-your-project.aivencloud.com:13041/bootstrap):\n 3 brokers:\n broker 2 at 35.205.234.70:13041\n broker 3 at 34.77.127.70:13041 (controller)\n broker 1 at 34.78.146.156:13041\n 0 topics:\nKafkaTopic and KafkaACL","text":"To properly produce and consume content on Kafka, you need topics and ACLs. The operator supports both with the KafkaTopic and KafkaACL resources.
Below, here is how to create a Kafka topic named random-strings where random string messages will be sent.
1. Create a file named kafka-topic-random-strings.yaml with the content below:
apiVersion: aiven.io/v1alpha1\nkind: KafkaTopic\nmetadata:\n name: random-strings\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n project: PROJECT_NAME\n serviceName: kafka-sample\n\n # here we can specify how many partitions the topic should have\n partitions: 3\n # and the topic replication factor\n replication: 2\n\n # we also support various topic-specific configurations\n config:\n flush_ms: 100\n2. Create the resource on Kubernetes:
kubectl apply -f kafka-topic-random-strings.yaml\n3. Create a user and an ACL. To use the Kafka topic, create a new user with the ServiceUser resource (in order to avoid using the avnadmin superuser), and the KafkaACL to allow the user access to the topic.
In a file named kafka-acl-user-crab.yaml, add the following two resources:
apiVersion: aiven.io/v1alpha1\nkind: ServiceUser\nmetadata:\n # the name of our user \ud83e\udd80\n name: crab\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n # the Secret name we will store the users' connection information\n # looks exactly the same as the Secret generated when creating the Kafka cluster\n # we will use this Secret to produce and consume events later!\n connInfoSecretTarget:\n name: kafka-crab-connection\n\n # the Aiven project the user is related to\n project: PROJECT_NAME\n\n # the name of our Kafka Service\n serviceName: kafka-sample\n\n---\napiVersion: aiven.io/v1alpha1\nkind: KafkaACL\nmetadata:\n name: crab\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n project: PROJECT_NAME\n serviceName: kafka-sample\n\n # the username from the ServiceUser above\n username: crab\n\n # the ACL allows to produce and consume on the topic\n permission: readwrite\n\n # specify the topic we created before\n topic: random-strings\nTo create the crab user and its permissions, execute the following command:
kubectl apply -f kafka-acl-user-crab.yaml\nUsing the previously created KafkaTopic, ServiceUser, KafkaACL, you can produce and consume events.
You can use kcat to produce a message into Kafka, and the -t random-strings argument to select the desired topic, and use the content of the /etc/issue file as the message's body.
1. Create a kafka-crab-produce.yaml file with the content below:
apiVersion: v1\nkind: Pod\nmetadata:\n name: kafka-crab-produce\nspec:\n restartPolicy: Never\n containers:\n - image: edenhill/kcat:1.7.0\n name: kcat\n\n # the command below will produce a message with the /etc/issue file content\n command:\n [\n \"kcat\",\n \"-b\",\n \"$(HOST):$(PORT)\",\n \"-X\",\n \"security.protocol=SSL\",\n \"-X\",\n \"ssl.key.location=/crab-auth/ACCESS_KEY\",\n \"-X\",\n \"ssl.key.password=$(PASSWORD)\",\n \"-X\",\n \"ssl.certificate.location=/crab-auth/ACCESS_CERT\",\n \"-X\",\n \"ssl.ca.location=/crab-auth/CA_CERT\",\n \"-P\",\n \"-t\",\n \"random-strings\",\n \"/etc/issue\",\n ]\n\n # loading the crab user data from the Secret as environment variables\n # useful to access the Kafka information, like hostname and port\n envFrom:\n - secretRef:\n name: kafka-crab-connection\n\n volumeMounts:\n - name: crab-auth\n mountPath: \"/crab-auth\"\n\n # loading the crab user information from the Secret as files in a volume\n # useful to access the Kafka certificates\n volumes:\n - name: crab-auth\n secret:\n secretName: kafka-crab-connection\n2. Create the Pod with the following content:
kubectl apply -f kafka-crab-produce.yaml\nNow your event is stored in Kafka.
To consume a message, you can use a graphical interface called Kowl. It allows you to explore information about our Kafka cluster, such as brokers, topics, or consumer groups.
1. Create a Kubernetes Pod and service to deploy and access Kowl. Create a file named kafka-crab-consume.yaml with the content below:
apiVersion: v1\nkind: Pod\nmetadata:\n name: kafka-crab-consume\n labels:\n app: kafka-crab-consume\nspec:\n containers:\n - image: quay.io/cloudhut/kowl:v1.4.0\n name: kowl\n\n # kowl configuration values\n env:\n - name: KAFKA_TLS_ENABLED\n value: \"true\"\n\n - name: KAFKA_BROKERS\n value: $(HOST):$(PORT)\n - name: KAFKA_TLS_PASSPHRASE\n value: $(PASSWORD)\n\n - name: KAFKA_TLS_CAFILEPATH\n value: /crab-auth/CA_CERT\n - name: KAFKA_TLS_CERTFILEPATH\n value: /crab-auth/ACCESS_CERT\n - name: KAFKA_TLS_KEYFILEPATH\n value: /crab-auth/ACCESS_KEY\n\n # inject all connection information as environment variables\n envFrom:\n - secretRef:\n name: kafka-crab-connection\n\n volumeMounts:\n - name: crab-auth\n mountPath: /crab-auth\n\n # loading the crab user information from the Secret as files in a volume\n # useful to access the Kafka certificates\n volumes:\n - name: crab-auth\n secret:\n secretName: kafka-crab-connection\n\n---\n# we will be using a simple service to access Kowl on port 8080\napiVersion: v1\nkind: Service\nmetadata:\n name: kafka-crab-consume\nspec:\n selector:\n app: kafka-crab-consume\n ports:\n - port: 8080\n targetPort: 8080\n2. Create the resources with:
kubectl apply -f kafka-crab-consume.yaml\n3. In another terminal create a port-forward tunnel to your Pod:
kubectl port-forward kafka-crab-consume 8080:8080\n4. In the browser of your choice, access the http://localhost:8080 address. You now see a page with the random-strings topic listed:
5. Click the topic name to see the message.
You have now consumed the message.
"},{"location":"resources/kafka/connect.html","title":"Kafka Connect","text":"Aiven for Apache Kafka Connect is a framework and a runtime for integrating Kafka with other systems. Kafka connectors can either be a source (for pulling data from other systems into Kafka) or sink (for pushing data into other systems from Kafka).
This section involves a few different Kubernetes CRDs:
KafkaService service with a KafkaTopicKafkaConnect serviceServiceIntegration to integrate the Kafka and KafkaConnect servicesPostgreSQL used as a sink to receive messages from KafkaKafkaConnector to finally connect the Kafka with the PostgreSQLCreate a file named kafka-sample-connect.yaml with the following content:
apiVersion: aiven.io/v1alpha1\nkind: Kafka\nmetadata:\n name: kafka-sample-connect\nspec:\n # gets the authentication token from the `aiven-token` Secret\n authSecretRef:\n name: aiven-token\n key: token\n\n # outputs the Kafka connection on the `kafka-connection` Secret\n connInfoSecretTarget:\n name: kafka-auth\n\n # add your Project name here\n project: PROJECT_NAME\n\n # cloud provider and plan of your choice\n # you can check all of the possibilities here https://aiven.io/pricing\n cloudName: google-europe-west1\n plan: business-4\n\n # general Aiven configuration\n maintenanceWindowDow: friday\n maintenanceWindowTime: 23:00:00\n\n # specific Kafka configuration\n userConfig:\n kafka_version: \"2.7\"\n kafka_connect: true\n\n---\napiVersion: aiven.io/v1alpha1\nkind: KafkaTopic\nmetadata:\n name: kafka-topic-connect\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n project: PROJECT_NAME\n serviceName: kafka-sample-connect\n\n replication: 2\n partitions: 1\nNext, create a file named kafka-connect.yaml and add the following KafkaConnect resource:
apiVersion: aiven.io/v1alpha1\nkind: KafkaConnect\nmetadata:\n name: kafka-connect\nspec:\n # gets the authentication token from the `aiven-token` Secret\n authSecretRef:\n name: aiven-token\n key: token\n\n # add your Project name here\n project: PROJECT_NAME\n\n # cloud provider and plan of your choice\n # you can check all of the possibilities here https://aiven.io/pricing\n cloudName: google-europe-west1\n plan: startup-4\n\n # general Aiven configuration\n maintenanceWindowDow: friday\n maintenanceWindowTime: 23:00:00\nNow let's create a ServiceIntegration. It will use the fields sourceServiceName and destinationServiceName to integrate the previously created kafka-sample-connect and kafka-connect. Open a new file named service-integration-connect.yaml and add the content below:
apiVersion: aiven.io/v1alpha1\nkind: ServiceIntegration\nmetadata:\n name: service-integration-kafka-connect\nspec:\n # gets the authentication token from the `aiven-token` Secret\n authSecretRef:\n name: aiven-token\n key: token\n\n project: PROJECT_NAME\n\n # indicates the type of the integration\n integrationType: kafka_connect\n\n # we will send messages from the `kafka-sample-connect` to `kafka-connect`\n sourceServiceName: kafka-sample-connect\n destinationServiceName: kafka-connect\nLet's add an Aiven for PostgreSQL service. It will be the service used as a sink, receiving messages from the kafka-sample-connect cluster. Create a file named pg-sample-connect.yaml with the content below:
apiVersion: aiven.io/v1alpha1\nkind: PostgreSQL\nmetadata:\n name: pg-connect\nspec:\n # gets the authentication token from the `aiven-token` Secret\n authSecretRef:\n name: aiven-token\n key: token\n\n # outputs the PostgreSQL connection on the `pg-connection` Secret\n connInfoSecretTarget:\n name: pg-connection\n\n # add your Project name here\n project: PROJECT_NAME\n\n # cloud provider and plan of your choice\n # you can check all of the possibilities here https://aiven.io/pricing\n cloudName: google-europe-west1\n plan: startup-4\n\n # general Aiven configuration\n maintenanceWindowDow: friday\n maintenanceWindowTime: 23:00:00\nFinally, let's add the glue of everything: a KafkaConnector. As described in the specification, it will send receive messages from the kafka-sample-connect and send them to the pg-connect service. Check our official documentation for more connectors.
Create a file named kafka-connector-connect.yaml and with the content below:
apiVersion: aiven.io/v1alpha1\nkind: KafkaConnector\nmetadata:\n name: kafka-connector\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n project: PROJECT_NAME\n\n # the Kafka cluster name\n serviceName: kafka-sample-connect\n\n # the connector we will be using\n connectorClass: io.aiven.connect.jdbc.JdbcSinkConnector\n\n userConfig:\n auto.create: \"true\"\n\n # constructs the pg-connect connection information\n connection.url: 'jdbc:postgresql://{{ fromSecret \"pg-connection\" \"PGHOST\"}}:{{ fromSecret \"pg-connection\" \"PGPORT\" }}/{{ fromSecret \"pg-connection\" \"PGDATABASE\" }}'\n connection.user: '{{ fromSecret \"pg-connection\" \"PGUSER\" }}'\n connection.password: '{{ fromSecret \"pg-connection\" \"PGPASSWORD\" }}'\n\n # specify which topics it will watch\n topics: kafka-topic-connect\n\n key.converter: org.apache.kafka.connect.json.JsonConverter\n value.converter: org.apache.kafka.connect.json.JsonConverter\n value.converter.schemas.enable: \"true\"\nWith all the files created, apply the new Kubernetes resources:
kubectl apply \\\n -f kafka-sample-connect.yaml \\\n -f kafka-connect.yaml \\\n -f service-integration-connect.yaml \\\n -f pg-sample-connect.yaml \\\n -f kafka-connector-connect.yaml\nIt will take some time for all the services to be up and running. You can check their status with the following command:
kubectl get \\\n kafkas.aiven.io/kafka-sample-connect \\\n kafkaconnects.aiven.io/kafka-connect \\\n postgresqls.aiven.io/pg-connect \\\n kafkaconnectors.aiven.io/kafka-connector\nThe output is similar to the following:
NAME PROJECT REGION PLAN STATE\nkafka.aiven.io/kafka-sample-connect your-project google-europe-west1 business-4 RUNNING\n\nNAME STATE\nkafkaconnect.aiven.io/kafka-connect RUNNING\n\nNAME PROJECT REGION PLAN STATE\npostgresql.aiven.io/pg-connect your-project google-europe-west1 startup-4 RUNNING\n\nNAME SERVICE NAME PROJECT CONNECTOR CLASS STATE TASKS TOTAL TASKS RUNNING\nkafkaconnector.aiven.io/kafka-connector kafka-sample-connect your-project io.aiven.connect.jdbc.JdbcSinkConnector RUNNING 1 1\nThe deployment is finished when all services have the state RUNNING.
To test the connection integration, let's produce a Kafka message using kcat from within the Kubernetes cluster. We will deploy a Pod responsible for crafting a message and sending to the Kafka cluster, using the kafka-auth secret generate by the Kafka CRD.
Create a new file named kcat-connect.yaml and add the content below:
apiVersion: v1\nkind: Pod\nmetadata:\n name: kafka-message\nspec:\n containers:\n\n restartPolicy: Never\n - image: edenhill/kcat:1.7.0\n name: kcat\n\n command: ['/bin/sh']\n args: [\n '-c',\n 'echo {\\\"schema\\\":{\\\"type\\\":\\\"struct\\\",\\\"fields\\\":[{ \\\"field\\\": \\\"text\\\", \\\"type\\\": \\\"string\\\", \\\"optional\\\": false } ] }, \\\"payload\\\": { \\\"text\\\": \\\"Hello World\\\" } } > /tmp/msg;\n\n kcat\n -b $(HOST):$(PORT)\n -X security.protocol=SSL\n -X ssl.key.location=/kafka-auth/ACCESS_KEY\n -X ssl.key.password=$(PASSWORD)\n -X ssl.certificate.location=/kafka-auth/ACCESS_CERT\n -X ssl.ca.location=/kafka-auth/CA_CERT\n -P -t kafka-topic-connect /tmp/msg'\n ]\n\n envFrom:\n - secretRef:\n name: kafka-auth\n\n volumeMounts:\n - name: kafka-auth\n mountPath: \"/kafka-auth\"\n\n volumes:\n - name: kafka-auth\n secret:\n secretName: kafka-auth\nApply the file with:
kubectl apply -f kcat-connect.yaml\nThe Pod will execute the commands and finish. You can confirm its Completed state with:
kubectl get pod kafka-message\nThe output is similar to the following:
NAME READY STATUS RESTARTS AGE\nkafka-message 0/1 Completed 0 5m35s\nIf everything went smoothly, we should have our produced message in the PostgreSQL service. Let's check that out.
Create a file named psql-connect.yaml with the content below:
apiVersion: v1\nkind: Pod\nmetadata:\n name: psql-connect\nspec:\n restartPolicy: Never\n containers:\n - image: postgres:13\n name: postgres\n # \"kafka-topic-connect\" is the table automatically created by KafkaConnect\n command:\n [\n \"psql\",\n \"$(DATABASE_URI)\",\n \"-c\",\n 'SELECT * from \"kafka-topic-connect\";',\n ]\n\n envFrom:\n - secretRef:\n name: pg-connection\nApply the file with:
kubectl apply -f psql-connect.yaml\nAfter a couple of seconds, inspect its log with this command:
kubectl logs psql-connect\nThe output is similar to the following:
text\n-------------\n Hello World\n(1 row)\nTo clean up all the created resources, use the following command:
kubectl delete \\\n -f kafka-sample-connect.yaml \\\n -f kafka-connect.yaml \\\n -f service-integration-connect.yaml \\\n -f pg-sample-connect.yaml \\\n -f kafka-connector-connect.yaml \\\n -f kcat-connect.yaml \\\n -f psql-connect.yaml\nKafkaSchema","text":"Aiven develops and maintain Karapace, an open source implementation of Kafka REST and schema registry. Is available out of the box for our managed Kafka service.
The schema registry address and authentication is the same as the Kafka broker, the only different is the usage of the port 13044.
First, let's create an Aiven for Apache Kafka service.
1. Create a file named kafka-sample-schema.yaml and add the content below:
apiVersion: aiven.io/v1alpha1\nkind: Kafka\nmetadata:\n name: kafka-sample-schema\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n connInfoSecretTarget:\n name: kafka-auth\n\n project: PROJECT_NAME\n cloudName: google-europe-west1\n plan: startup-2\n maintenanceWindowDow: friday\n maintenanceWindowTime: 23:00:00\n\n userConfig:\n kafka_version: \"2.7\"\n\n # this flag enables the Schema registry\n schema_registry: true\n2. Apply the changes with the following command:
kubectl apply -f kafka-schema.yaml\nNow, let's create the schema itself.
1. Create a new file named kafka-sample-schema.yaml and add the YAML content below:
apiVersion: aiven.io/v1alpha1\nkind: KafkaSchema\nmetadata:\n name: kafka-schema\nspec:\n authSecretRef:\n name: aiven-token\n key: token\n\n project: PROJECT_NAME\n serviceName: kafka-sample-schema\n\n # the name of the Schema\n subjectName: MySchema\n\n # the schema itself, in JSON format\n schema: |\n {\n \"type\": \"record\",\n \"name\": \"MySchema\",\n \"fields\": [\n {\n \"name\": \"field\",\n \"type\": \"string\"\n }\n ]\n }\n\n # sets the schema compatibility level\n compatibilityLevel: BACKWARD\n2. Create the schema with the command:
kubectl apply -f kafka-schema.yaml\n3. Review the resource you created with the following command:
kubectl get kafkaschemas.aiven.io kafka-schema\nThe output is similar to the following:
NAME SERVICE NAME PROJECT SUBJECT COMPATIBILITY LEVEL VERSION\nkafka-schema kafka-sample PROJECT_NAME MySchema BACKWARD 1\nNow you can follow the instructions to use a schema registry in Java on how to use the schema created.
"}]} \ No newline at end of file diff --git a/sitemap.xml b/sitemap.xml index be93285f..ef2cc9b4 100644 --- a/sitemap.xml +++ b/sitemap.xml @@ -2,247 +2,247 @@