Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

OPS-6220-Ionos-Exporter Extension #4

Open
wants to merge 56 commits into
base: main
Choose a base branch
from
Open
Changes from 1 commit
Commits
Show all changes
56 commits
Select commit Hold shift + click to select a range
75cbc31
implementation for s3 metrics and ipmetrics
aebyss May 2, 2024
3043a1e
initial push
aebyss May 2, 2024
367c7d4
fixed problems with crashing while using go routine
aebyss May 2, 2024
672a5d7
x
aebyss May 2, 2024
b07829e
Wait groups work in progress
aebyss May 6, 2024
f3568dd
implemented concurrency for s3 log parsing, fixed bugs, implemented a…
aebyss May 23, 2024
3a3e559
fixing github action
aebyss May 23, 2024
2c23564
moved GetHeadBucket from local file to helper.go
aebyss May 23, 2024
2222a26
fixed declared and not used error
aebyss May 23, 2024
cf7f08b
added go mod tidy for installing dependencies
aebyss May 23, 2024
1bc7e18
removed some of log Statements and added secret management for s3
aebyss May 23, 2024
b1b2614
resotred CollectResources func in main
aebyss May 24, 2024
f7abecc
reverted api cycle to 900
aebyss May 24, 2024
73d98ac
removed logging to a file
aebyss May 24, 2024
8ea6e5d
added new labels removed some comments
aebyss May 28, 2024
3511721
Fix typo
simoncolincap May 29, 2024
0850549
fixed scraping inconsistencies of metrics and added error handling fo…
aebyss May 31, 2024
b3e2be1
Merge branch 'DBP-ionos-exporter-expansion' of https://github.com/dBi…
aebyss May 31, 2024
3b43466
added scraping of bucket tags and pushing them to prometheus
aebyss Jun 13, 2024
eb30380
added git.sum file
aebyss Jun 13, 2024
c09f3af
removed customcollector.go
aebyss Jun 13, 2024
283b5a1
added error handling for number of IPs
aebyss Jun 21, 2024
8a1f92d
Added logger instead of print and changed number of ips from GaugeVec…
aebyss Jun 24, 2024
007e640
Added postgress cluster metrics and telemetry metrics
aebyss Jul 2, 2024
298524d
refactoring prometheus.go@
aebyss Jul 3, 2024
f063b1f
Bug for when bucket doesnt have tags wont be pushed to prometheus fixed
aebyss Jul 4, 2024
663ba85
Added Config.yaml
aebyss Jul 4, 2024
675ba61
temp removal of postgress collection before i configure the deploymen…
aebyss Jul 4, 2024
a2f7255
s3 refactor and ConfigMap definition
aebyss Jul 5, 2024
f184409
fixed configmap locatioN
aebyss Jul 5, 2024
064855c
fixed syntax for helm deployment
aebyss Jul 5, 2024
6ef9510
removed print statements used for debugging
aebyss Jul 5, 2024
dde6c49
syntax fixes and added comments to the ionos_scrapper functions
aebyss Jul 9, 2024
38039d1
fixed config path loader
aebyss Jul 15, 2024
ede28cd
added documentation and removed redundant code
aebyss Aug 1, 2024
9a79289
removed local loading of .env
aebyss Aug 2, 2024
289725d
removed .env in postgres_scraper
aebyss Aug 2, 2024
defffda
debugging s3 function not loading in pods
aebyss Aug 2, 2024
41e0088
ommited usage of config.yaml for s3 scraping
aebyss Aug 2, 2024
078fa1f
syntax fixes, some commets removed
aebyss Aug 5, 2024
4e075c4
added buffer to reading lines
aebyss Aug 6, 2024
3bdbead
added config changes and added env template file for running localy
aebyss Aug 9, 2024
fbd3252
added path to config in deploymenet yaml
aebyss Aug 9, 2024
f52ed80
added default path to config.yaml
aebyss Aug 9, 2024
9bf9330
removed default path to config and removed os.Exit when n oconfig set
aebyss Aug 9, 2024
5ec471d
trying gracefully skipping local setting of config yaml
aebyss Aug 9, 2024
ab7cdda
syntax fix
aebyss Aug 9, 2024
303846b
fixed nil dereference problem
aebyss Aug 9, 2024
cdef725
added default path again
aebyss Aug 9, 2024
f39fa28
removed line configfile as enviroment variable only as parameter is n…
aebyss Aug 14, 2024
f256591
Removed cost-calc sequenqce diagramm from Documentation
aebyss Aug 14, 2024
f9e9fab
make s3 part optional
JannikBadenhop Oct 10, 2024
7f03d99
Fix if, Name -> Owner
JannikBadenhop Oct 10, 2024
d9986d3
increase minor version and reset patch version, since values and defa…
JannikBadenhop Oct 10, 2024
b06d005
revert version changes
JannikBadenhop Oct 10, 2024
7c6aeba
fix quoting and whitespace issues
JannikBadenhop Oct 11, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Added postgress cluster metrics and telemetry metrics
aebyss committed Jul 2, 2024
commit 007e640c2fdd901f8469de824f6ef87fea2feadd
50 changes: 50 additions & 0 deletions config.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,50 @@
tenants:
- name: spsh
operations:
- clusters
- databases
- name: dbildungs
operations:
- clusters


metrics:
- name: ionos_dbaas_postgres_transactions:rate2m
description: Per-second average rate of SQL transactions (that have been committed), calculated over the last 2 minutes.
type: gauge
- name: ionos_dbaas_postgres_connections_count
description: Number of connections per instance and state. active, disabled, fastpath function call, idle, idle in transaction, idle in transaction (aborted).
type: gauge
- name: ionos_dbaas_postgres_cpu_rate5m
description: The average CPU utilization over the past 5 minutes.
type: gauge
- name: ionos_dbaas_postgres_disk_io_time_weighted_seconds_rate5m
description: The rate of disk I/O time, in seconds, over a five-minute period. Provides insight into performance of a disk, as high values may indicate that the disk is being overused or is experiencing performance issues.
type: gauge
- name: ionos_dbaas_postgres_instance_count
description: Desired number of instances. The number of currently ready and running instances may be different. ionos_dbaas_postgres_role provides information about running instances split by role.
type: gauge
- name: ionos_dbaas_postgres_load5
description: Linux load average for the last 5 minutes. This metric is represented as a number between 0 and 1 (can be greater than 1 on multicore machines), where 0 indicates that the CPU core is idle and 1 indicates that the CPU core is fully utilized. Higher values may indicate that the system is experiencing performance issues or is approaching capacity.
type: gauge
- name: ionos_dbaas_postgres_memory_available_bytes
description: Available memory in bytes.
type: gauge
- name: ionos_dbaas_postgres_memory_total_bytes
description: Total memory of the underlying machine in bytes. Some of it is used for our management and monitoring tools and not available to PostgreSQL. During horizontal scaling you might see different values for each instance.
type: gauge
- name: ionos_dbaas_postgres_role
description: Current role of the instance. Provides whether an instance is currently "master" or "replica".
type: gauge
- name: ionos_dbaas_postgres_storage_available_bytes
description: Free available disk space per instance in bytes.
type: gauge
- name: ionos_dbaas_postgres_storage_total_bytes
description: Total disk space per instance in bytes. During horizontal scaling you might see different values for each instance.
type: gauge
- name: ionos_dbaas_postgres_user_tables_idx_scan
description: Number of index scans per table/schema.
type: gauge
- name: ionos_dbaas_postgres_user_tables_seq_scan
description: Number of sequential scans per table/schema. A high number of sequential scans may indicate that an index should be added to improve performance.
type: gauge
6 changes: 6 additions & 0 deletions go.mod
Original file line number Diff line number Diff line change
@@ -13,6 +13,9 @@ require (
github.com/beorn7/perks v1.0.1 // indirect
github.com/cespare/xxhash/v2 v2.2.0 // indirect
github.com/golang/protobuf v1.5.3 // indirect
github.com/ionos-cloud/sdk-go-bundle/products/dbaas/psql v0.1.0 // indirect
github.com/ionos-cloud/sdk-go-bundle/shared v0.1.0 // indirect
github.com/ionos-cloud/sdk-go-dbaas-postgres v1.1.2 // indirect
github.com/jmespath/go-jmespath v0.4.0 // indirect
github.com/matttproud/golang_protobuf_extensions v1.0.4 // indirect
github.com/prometheus/client_model v0.3.0 // indirect
@@ -23,4 +26,7 @@ require (
golang.org/x/sys v0.21.0 // indirect
google.golang.org/appengine v1.6.7 // indirect
google.golang.org/protobuf v1.30.0 // indirect
gopkg.in/alexcesaro/quotedprintable.v3 v3.0.0-20150716171945-2caba252f4dc // indirect
gopkg.in/gomail.v2 v2.0.0-20160411212932-81ebce5c23df // indirect
gopkg.in/yaml.v2 v2.4.0 // indirect
)
11 changes: 11 additions & 0 deletions go.sum
Original file line number Diff line number Diff line change
@@ -14,6 +14,12 @@ github.com/golang/protobuf v1.5.3 h1:KhyjKVUg7Usr/dYsdSqoFveMYd5ko72D+zANwlG1mmg
github.com/golang/protobuf v1.5.3/go.mod h1:XVQd3VNwM+JqD3oG2Ue2ip4fOMUkwXdXDdiuN0vRsmY=
github.com/google/go-cmp v0.5.5/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
github.com/google/go-cmp v0.5.9 h1:O2Tfq5qg4qc4AmwVlvv0oLiVAGB7enBSJ2x2DqQFi38=
github.com/ionos-cloud/sdk-go-bundle/products/dbaas/psql v0.1.0 h1:s/FZdHaews1vIRJYbJUvaO+Y9yYIiJ1z+s1HLMcCii0=
github.com/ionos-cloud/sdk-go-bundle/products/dbaas/psql v0.1.0/go.mod h1:8qebNZf4GeZm64BDUqeunDDw/jIQqhry6RyIhEqIHJE=
github.com/ionos-cloud/sdk-go-bundle/shared v0.1.0 h1:1psLbSn+i/wK0Samy8XqLxsqEnhTshz3Py1lbp5F/ao=
github.com/ionos-cloud/sdk-go-bundle/shared v0.1.0/go.mod h1:hDQ5fhujYFaYr1fD8bmmf4rlCKAsqKwczuBS0Z5PNBQ=
github.com/ionos-cloud/sdk-go-dbaas-postgres v1.1.2 h1:AaKbci+kVS6/k43VwJwmXxCJ7pzj9jwuOPqO8Wd5560=
github.com/ionos-cloud/sdk-go-dbaas-postgres v1.1.2/go.mod h1:nmJEwuRX65A5/PxwvdFW0XrV+N6WFYnMV1TiIafAwz4=
github.com/ionos-cloud/sdk-go/v6 v6.1.9 h1:Iq3VIXzeEbc8EbButuACgfLMiY5TPVWUPNrF+Vsddo4=
github.com/ionos-cloud/sdk-go/v6 v6.1.9/go.mod h1:EzEgRIDxBELvfoa/uBN0kOQaqovLjUWEB7iW4/Q+t4k=
github.com/jmespath/go-jmespath v0.4.0 h1:BEgLn5cpjn8UN1mAw4NjwDrS35OdebyEtFe+9YPoQUg=
@@ -56,6 +62,11 @@ google.golang.org/protobuf v1.26.0-rc.1/go.mod h1:jlhhOSvTdKEhbULTjvd4ARK9grFBp0
google.golang.org/protobuf v1.26.0/go.mod h1:9q0QmTI4eRPtz6boOQmLYwt+qCgq0jsYwAQnmE0givc=
google.golang.org/protobuf v1.30.0 h1:kPPoIgf3TsEvrm0PFe15JQ+570QVxYzEvvHqChK+cng=
google.golang.org/protobuf v1.30.0/go.mod h1:HV8QOd/L58Z+nl8r43ehVNZIU/HEI6OcFqwMG9pJV4I=
gopkg.in/alexcesaro/quotedprintable.v3 v3.0.0-20150716171945-2caba252f4dc h1:2gGKlE2+asNV9m7xrywl36YYNnBG5ZQ0r/BOOxqPpmk=
gopkg.in/alexcesaro/quotedprintable.v3 v3.0.0-20150716171945-2caba252f4dc/go.mod h1:m7x9LTH6d71AHyAX77c9yqWCCa3UKHcVEj9y7hAtKDk=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/gomail.v2 v2.0.0-20160411212932-81ebce5c23df h1:n7WqCuqOuCbNr617RXOY0AWRXxgwEyPp2z+p0+hgMuE=
gopkg.in/gomail.v2 v2.0.0-20160411212932-81ebce5c23df/go.mod h1:LRQQ+SO6ZHR7tOkpBDuZnXENFzX8qRjMDMyPD6BRkCw=
gopkg.in/yaml.v2 v2.2.8/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
gopkg.in/yaml.v2 v2.4.0 h1:D8xgwECY7CYvx+Y2n4sBz93Jn9JRvxdiyyo8CTfuKaY=
gopkg.in/yaml.v2 v2.4.0/go.mod h1:RDklbk79AGWmwhnvt/jBztapEOGDOx6ZbXqjP6csGnQ=
258 changes: 258 additions & 0 deletions internal/postgres-test.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,258 @@
package internal

import (
"context"
"encoding/json"
"fmt"
"io/ioutil"
"log"
"net/http"
"os"
"sync"
"time"

psql "github.com/ionos-cloud/sdk-go-dbaas-postgres"
"github.com/joho/godotenv"
"gopkg.in/yaml.v2"
)

type Tenant struct {
Name string `yaml:"name"`
Operations []string `yaml:"operations"`
}

type IonosPostgresResources struct {
ClusterName string
CPU int32
RAM int32
Storage int32
Owner string
DatabaseNames []string
Telemetry []TelemetryMetric
}

type TelemetryMetric struct {
Metric map[string]string `json:"metric"`
Values [][]interface{} `json:"values"`
}

type TelemetryResponse struct {
Status string `json:status`
Data struct {
ResultType string `json:"resultType`
Result []TelemetryMetric `json:"result"`
} `json:"data"`
}

var (
ClusterCoresTotal int32 = 0
ClusterRamTotal int32 = 0
ClusterTotal int32 = 0
IonosPostgresClusters = make(map[string]IonosPostgresResources)
)

type Config struct {
Tenants []Tenant `yaml:"tenants"`
Metrics []Metric `yaml:"metrics"`
}

type Metric struct {
Name string `yaml:"name"`
Description string `yaml:"description"`
Type string `yaml:"type"`
}

func loadConfig(filename string) (*Config, error) {
data, err := ioutil.ReadFile(filename)
if err != nil {
return nil, err
}

var config Config
err = yaml.Unmarshal(data, &config)
if err != nil {
return nil, err
}

return &config, nil
}

func Testpsql(m *sync.RWMutex, cycletime int32) {
err := godotenv.Load(".env")
if err != nil {
fmt.Println("Error loading .env file")
}
cfgENV := psql.NewConfigurationFromEnv()

apiClient := psql.NewAPIClient(cfgENV)

config, err := loadConfig("config.yaml")
if err != nil {
log.Fatalf("Failed to load config: %v", err)
}

for {
var wg sync.WaitGroup
for _, tenant := range config.Tenants {
wg.Add(1)
go func(tenant Tenant) {
defer wg.Done()
processCluster(apiClient, m, config.Metrics)
}(tenant)
}
wg.Wait()
time.Sleep(time.Duration(cycletime) * time.Second)
}
}

func processCluster(apiClient *psql.APIClient, m *sync.RWMutex, metrics []Metric) {
datacenters, err := fetchClusters(apiClient)
if err != nil {
fmt.Fprintf(os.Stderr, "Failed to fetch clusters: %v\n", err)
}
newIonosPostgresResources := make(map[string]IonosPostgresResources)

for _, clusters := range *datacenters.Items {
databaseNames, err := fetchDatabases(apiClient, *clusters.Id)
if err != nil {
fmt.Fprintf(os.Stderr, "Failed to fetch databases for cluster %s: %v\n", *clusters.Properties.DisplayName, err)
continue
}
databaseOwner, err := fetchOwner(apiClient, *clusters.Id)
if err != nil {
fmt.Fprintf(os.Stderr, "Failed to fetch owner for database %s: %v\n", *clusters.Properties.DisplayName, err)
continue
}

telemetryData := make([]TelemetryMetric, 0)

for _, metricConfig := range metrics {
telemetryResp, err := fetchTelemetryMetrics(os.Getenv("IONOS_TOKEN"), fmt.Sprintf("%s{postgres_cluster=\"%s\"}", metricConfig.Name, *clusters.Id), *clusters.Id)
if err != nil {
fmt.Fprintf(os.Stderr, "Failed to fetch telemetry metrics for cluster %s: %v\n", *clusters.Id, err)
continue
}
telemetryData = append(telemetryData, telemetryResp.Data.Result...)
}

// fmt.Printf("Here are the database names %v", databaseNames)
newIonosPostgresResources[*clusters.Properties.DisplayName] = IonosPostgresResources{
ClusterName: *clusters.Properties.DisplayName,
CPU: *clusters.Properties.Cores,
RAM: *clusters.Properties.Ram,
Storage: *clusters.Properties.StorageSize,
DatabaseNames: databaseNames,
Owner: databaseOwner,
Telemetry: telemetryData,
}
}
// clusterName := "spsh-dev-schulportal"
m.Lock()
// fmt.Printf("Here is the telemetryData for cluster '%s': %v\n", clusterName, newIonosPostgresResources[clusterName].Telemetry)
// fmt.Printf("Here is the map %v", newIonosPostgresResources["telemetryData"])
IonosPostgresClusters = newIonosPostgresResources
m.Unlock()

}

func fetchClusters(apiClient *psql.APIClient) (*psql.ClusterList, error) {
datacenters, resp, err := apiClient.ClustersApi.ClustersGet(context.Background()).Execute()
if err != nil {
fmt.Fprintf(os.Stderr, "Error when calling ClustersApi: %v\n", err)
if resp != nil {
fmt.Fprintf(os.Stderr, "Full HTTP response: %v\n", resp)
} else {
fmt.Fprintf(os.Stderr, "No HTTP response received\n")
}
return nil, err
}

if datacenters.Items == nil {
return nil, fmt.Errorf("no items in resource")
}

return &datacenters, nil
}

func fetchDatabases(apiClient *psql.APIClient, clusterID string) ([]string, error) {
databases, resp, err := apiClient.DatabasesApi.DatabasesList(context.Background(), clusterID).Execute()
if err != nil {
fmt.Fprintf(os.Stderr, "Error when calling DatabasesApi: %v\n", err)
if resp != nil {
fmt.Fprintf(os.Stderr, "Full HTTP response: %v\n", resp)
} else {
fmt.Fprintf(os.Stderr, "No HTTP response received\n")
}
return nil, err
}

if databases.Items == nil {
return nil, fmt.Errorf("no databases found for cluster %s", clusterID)
}

var databaseNames []string

for _, db := range *databases.Items {
if db.Properties != nil && db.Properties.Name != nil {
databaseNames = append(databaseNames, *db.Properties.Name)
}
}
return databaseNames, nil
}

func fetchOwner(apiClient *psql.APIClient, clusterID string) (string, error) {
databases, resp, err := apiClient.DatabasesApi.DatabasesList(context.Background(), clusterID).Execute()
if err != nil {
fmt.Fprintf(os.Stderr, "Error when calling DatabasesApi: %v\n", err)
if resp != nil {
fmt.Fprintf(os.Stderr, "Full HTTP response: %v\n", resp)
} else {
fmt.Fprintf(os.Stderr, "No HTTP response received\n")
}
return "", err
}

if databases.Items == nil {
return "", fmt.Errorf("no databases found for cluster %s", clusterID)
}
var owner = ""
for _, db := range *databases.Items {
if db.Properties != nil && db.Properties.Name != nil {
owner = *db.Properties.Owner
}
}
return owner, nil
}

func fetchTelemetryMetrics(apiToken, query, clusterID string) (*TelemetryResponse, error) {
req, err := http.NewRequest("GET", "https://dcd.ionos.com/telemetry/api/v1/query_range", nil)
if err != nil {
return nil, err
}

q := req.URL.Query()
q.Add("query", query)
q.Add("start", time.Now().Add(-time.Hour).Format(time.RFC3339))
q.Add("end", time.Now().Format(time.RFC3339))
q.Add("step", "60")
req.URL.RawQuery = q.Encode()

req.Header.Set("Authorization", "Bearer "+apiToken)

client := &http.Client{}
resp, err := client.Do(req)
if err != nil {
return nil, err
}
defer resp.Body.Close()

var telemetryResp TelemetryResponse
if err := json.NewDecoder(resp.Body).Decode(&telemetryResp); err != nil {
fmt.Printf("Fialed to decode json response: %v\n", err)
return nil, err
}

// fmt.Printf("Telemetry Response: %+v\n", telemetryResp)

return &telemetryResp, nil
}
Loading