Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(kafka_quota): added support for kafka quota #1952

Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,10 @@ __debug_bin
# Output of the go coverage tool, specifically when used with LiteIDE
*.out

# Go workspace file
go.work
go.work.sum

# Dependency directories
vendor/
packrd/
Expand Down
5 changes: 5 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,11 @@ nav_order: 1
<!-- Always keep the following header in place: -->
<!--## [MAJOR.MINOR.PATCH] - YYYY-MM-DD -->


## [MAJOR.MINOR.PATCH] - YYYY-MM-DD

- Add `aiven_kafka_quota` resource

## [4.31.0] - 2024-12-18

- Add `alloydbomni` BETA resource and datasource
Expand Down
79 changes: 79 additions & 0 deletions docs/resources/kafka_quota.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,79 @@
---
# generated by https://github.com/hashicorp/terraform-plugin-docs
page_title: "aiven_kafka_quota Resource - terraform-provider-aiven"
subcategory: ""
description: |-
Creates and manages quotas for an Aiven for Apache Kafka® service user.
---

# aiven_kafka_quota (Resource)

Creates and manages quotas for an Aiven for Apache Kafka® service user.

## Example Usage

```terraform
resource "aiven_kafka_quota" "example_quota" {
project = data.aiven_project.foo.project
service_name = aiven_kafka.example_kafka.service_name
user = "example-kafka-user"
client_id = "example_client"
consumer_byte_rate = 1000
producer_byte_rate = 1000
request_percentage = 50
}
```

<!-- schema generated by tfplugindocs -->
## Schema

### Required

- `project` (String) The name of the project this resource belongs to. To set up proper dependencies please refer to this variable as a reference. Changing this property forces recreation of the resource.
- `service_name` (String) The name of the service that this resource belongs to. To set up proper dependencies please refer to this variable as a reference. Changing this property forces recreation of the resource.

### Optional

- `client_id` (String) Represents a logical group of clients, assigned a unique name by the client application.
Quotas can be applied based on user, client-id, or both.
The most relevant quota is chosen for each connection.
All connections within a quota group share the same quota.
It is possible to set default quotas for each (user, client-id), user or client-id group by specifying 'default'
- `consumer_byte_rate` (Number) Defines the bandwidth limit in bytes/sec for each group of clients sharing a quota.
Every distinct client group is allocated a specific quota, as defined by the cluster, on a per-broker basis.
Exceeding this limit results in client throttling.
- `producer_byte_rate` (Number) Defines the bandwidth limit in bytes/sec for each group of clients sharing a quota.
Every distinct client group is allocated a specific quota, as defined by the cluster, on a per-broker basis.
Exceeding this limit results in client throttling.
- `request_percentage` (Number) Sets the maximum percentage of CPU time that a client group can use on request handler I/O and network threads per broker within a quota window.
Exceeding this limit triggers throttling.
The quota, expressed as a percentage, also indicates the total allowable CPU usage for the client groups sharing the quota.
- `timeouts` (Block, Optional) (see [below for nested schema](#nestedblock--timeouts))
- `user` (String) Represents a logical group of clients, assigned a unique name by the client application.
Quotas can be applied based on user, client-id, or both.
The most relevant quota is chosen for each connection.
All connections within a quota group share the same quota.
It is possible to set default quotas for each (user, client-id), user or client-id group by specifying 'default'

### Read-Only

- `id` (String) The ID of this resource.

<a id="nestedblock--timeouts"></a>
### Nested Schema for `timeouts`

Optional:

- `create` (String)
- `default` (String)
- `delete` (String)
- `read` (String)
- `update` (String)

## Import

Import is supported using the following syntax:

```shell
terraform import aiven_kafka_quota.example_quota PROJECT/SERVICE_NAME
```
1 change: 1 addition & 0 deletions examples/resources/aiven_kafka_quota/import.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
terraform import aiven_kafka_quota.example_quota PROJECT/SERVICE_NAME
9 changes: 9 additions & 0 deletions examples/resources/aiven_kafka_quota/resource.tf
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
resource "aiven_kafka_quota" "example_quota" {
project = data.aiven_project.foo.project
service_name = aiven_kafka.example_kafka.service_name
user = "example-kafka-user"
client_id = "example_client"
consumer_byte_rate = 1000
producer_byte_rate = 1000
request_percentage = 50
}
2 changes: 1 addition & 1 deletion go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ go 1.23

require (
github.com/aiven/aiven-go-client/v2 v2.33.0
github.com/aiven/go-client-codegen v0.71.0
github.com/aiven/go-client-codegen v0.73.0
github.com/avast/retry-go v3.0.0+incompatible
github.com/dave/jennifer v1.7.1
github.com/docker/go-units v0.5.0
Expand Down
2 changes: 2 additions & 0 deletions go.sum
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,8 @@ github.com/aiven/go-api-schemas v1.106.0 h1:qncRsbiaGnU9JE9fmTFHclTCBem+t+6EPMXG
github.com/aiven/go-api-schemas v1.106.0/go.mod h1:z7dGvufm6If4gOdVr7dWTuFZmll9FOZr5Z5CSxGpebA=
github.com/aiven/go-client-codegen v0.71.0 h1:SGiHrfbU8RiqVegQGV3BStnbIdFke+15lxadlPORqfI=
github.com/aiven/go-client-codegen v0.71.0/go.mod h1:QKN/GgLMGWd6+gPEucXlZPi5vC3C6RpD3UeBRQOLI1Y=
github.com/aiven/go-client-codegen v0.73.0 h1:1xk7zmAqKxQYHWE4ARWFlKHZg8FB4VTDGxVua7iruRg=
github.com/aiven/go-client-codegen v0.73.0/go.mod h1:QKN/GgLMGWd6+gPEucXlZPi5vC3C6RpD3UeBRQOLI1Y=
github.com/apparentlymart/go-textseg/v12 v12.0.0/go.mod h1:S/4uRK2UtaQttw1GenVJEynmyUenKwP++x/+DdGV/Ec=
github.com/apparentlymart/go-textseg/v15 v15.0.0 h1:uYvfpb3DyLSCGWnctWKGj857c6ew1u1fNQOlOtuGxQY=
github.com/apparentlymart/go-textseg/v15 v15.0.0/go.mod h1:K8XmNZdhEBkdlyDdvbmmsvpAG721bKi0joRfFdHIWJ4=
Expand Down
223 changes: 223 additions & 0 deletions internal/acctest/template.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,223 @@
package acctest

import (
"bytes"
"fmt"
"html/template"
"sort"
"strings"
"testing"
)

// ResourceConfig is the interface that all resource configs must implement
type resourceConfig interface {
// ToMap converts the config to a map for template rendering
ToMap() map[string]any
}

// Template represents a single Terraform configuration template
type Template struct {
Name string
Template string
}

// TemplateRegistry holds templates for a specific resource type
type TemplateRegistry struct {
resourceName string
templates map[string]*template.Template
funcMap template.FuncMap
}

// NewTemplateRegistry creates a new template registry for a resource
func NewTemplateRegistry(resourceName string) *TemplateRegistry {
return &TemplateRegistry{
resourceName: resourceName,
templates: make(map[string]*template.Template),
funcMap: make(template.FuncMap),
}
}

// AddTemplate adds a new template to the registry
func (r *TemplateRegistry) AddTemplate(t testing.TB, name, templateStr string) error {
t.Helper()

tmpl := template.New(name)
if len(r.funcMap) > 0 {
tmpl = tmpl.Funcs(r.funcMap)
}

parsed, err := tmpl.Parse(templateStr)
if err != nil {
return fmt.Errorf("failed to parse template: %w", err)
}
r.templates[name] = parsed

return nil
}

// MustAddTemplate is like AddTemplate but panics on error
func (r *TemplateRegistry) MustAddTemplate(t testing.TB, name, templateStr string) {
t.Helper()

if err := r.AddTemplate(t, name, templateStr); err != nil {
t.Fatal(err)
}
}

// Render renders a template with the given config
func (r *TemplateRegistry) Render(t testing.TB, templateKey string, cfg map[string]any) (string, error) {
t.Helper()

tmpl, exists := r.templates[templateKey]
if !exists {
availableTemplates := r.getAvailableTemplates()

return "", fmt.Errorf("template %q does not exist for resource %s. Available templates: %v",
templateKey,
r.resourceName,
availableTemplates,
)
}

var buf bytes.Buffer
if err := tmpl.Execute(&buf, cfg); err != nil {
return "", fmt.Errorf("failed to render template: %w", err)
}

return buf.String(), nil
}

// MustRender is like Render but fails the test on error
func (r *TemplateRegistry) MustRender(t testing.TB, templateKey string, cfg map[string]any) string {
t.Helper()

result, err := r.Render(t, templateKey, cfg)
if err != nil {
t.Fatal(err)
}

return result
}

// AddFunction adds a custom function to the template registry
func (r *TemplateRegistry) AddFunction(name string, fn interface{}) {
if r.funcMap == nil {
r.funcMap = make(template.FuncMap)
}
r.funcMap[name] = fn
}

// HasTemplate checks if a template exists in the registry
func (r *TemplateRegistry) HasTemplate(key string) bool {
_, exists := r.templates[key]
return exists
}

// RemoveTemplate removes a template from the registry
func (r *TemplateRegistry) RemoveTemplate(key string) {
delete(r.templates, key)
}

// getAvailableTemplates returns a sorted list of available template keys
func (r *TemplateRegistry) getAvailableTemplates() []string {
templates := make([]string, 0, len(r.templates))
for k := range r.templates {
templates = append(templates, k)
}
sort.Strings(templates)

return templates
}

// compositionEntry represents a combination of template and its config
type compositionEntry struct {
TemplateKey string
Config map[string]any
}

// CompositionBuilder helps build complex compositions of templates
type CompositionBuilder struct {
registry *TemplateRegistry
compositions []compositionEntry
}

// NewCompositionBuilder creates a new composition builder
func (r *TemplateRegistry) NewCompositionBuilder() *CompositionBuilder {
return &CompositionBuilder{
registry: r,
compositions: make([]compositionEntry, 0),
}
}

// Add adds a new template and config to the composition
func (b *CompositionBuilder) Add(templateKey string, cfg map[string]any) *CompositionBuilder {
b.compositions = append(b.compositions, compositionEntry{
TemplateKey: templateKey,
Config: cfg,
})
return b
}

// AddWithConfig adds a new template and config to the composition using a resourceConfig
func (b *CompositionBuilder) AddWithConfig(templateKey string, cfg resourceConfig) *CompositionBuilder {
b.compositions = append(b.compositions, compositionEntry{
TemplateKey: templateKey,
Config: cfg.ToMap(),
})
return b
}

// AddIf conditional method to CompositionBuilder
func (b *CompositionBuilder) AddIf(condition bool, templateKey string, cfg map[string]any) *CompositionBuilder {
if condition {
return b.Add(templateKey, cfg)
}

return b
}

func (b *CompositionBuilder) Remove(templateKey string) *CompositionBuilder {
var newCompositions []compositionEntry
for _, comp := range b.compositions {
if comp.TemplateKey != templateKey {
newCompositions = append(newCompositions, comp)
}
}
b.compositions = newCompositions

return b
}

// Render renders all templates in the composition and combines them
func (b *CompositionBuilder) Render(t testing.TB) (string, error) {
t.Helper()

var renderedParts = make([]string, 0, len(b.compositions))

// Render each template
for _, comp := range b.compositions {
rendered, err := b.registry.Render(t, comp.TemplateKey, comp.Config)
if err != nil {
return "", fmt.Errorf("failed to render template %s: %w", comp.TemplateKey, err)
}
renderedParts = append(renderedParts, rendered)
}

// Combine all rendered parts
combined := strings.Join(renderedParts, "\n\n")

//TODO: add HCL validation?

return combined, nil
}

// MustRender is like Render but fails the test on error
func (b *CompositionBuilder) MustRender(t testing.TB) string {
t.Helper()

result, err := b.Render(t)
if err != nil {
t.Fatal(err)
}
return result
}
Loading
Loading