Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[kube-prometheus-stack]use metrics from this charts kubelet-service #4042

Closed
wants to merge 5 commits into from

Conversation

YuleZ
Copy link
Contributor

@YuleZ YuleZ commented Nov 27, 2023

What this PR does / why we need it

This PR adds the kubelet's service name from this helm-chart to prometheus-rules in order to avoid execution errors when using k8s-services that target kubelet as well.

In my case the elastic-search operator creates a similar service for Kubelet which errors in combination with kubelet-prometheus-rules from this helm-chart.

execution: 
found duplicate series for the match group 
{instance="x.x.x.x:10250"} on the right hand-side of the operation: 
[
{__name__="kubelet_node_name", endpoint="https-metrics", instance="x.x.x.x:10250", job="kubelet", metrics_path="/metrics", namespace="kube-system", node="node1", service="elastic-search-kubelet"},
{__name__="kubelet_node_name", endpoint="https-metrics", instance="x.x.x.x:10250", job="kubelet", metrics_path="/metrics", namespace="kube-system", node="node1", service="kube-prometheus-stack-kubelet"}
]
 ;many-to-many matching not allowed: matching labels must be unique on one side

Which issue this PR fixes

See above

Special notes for your reviewer

Checklist

  • DCO signed
  • Chart Version bumped
  • Title of the PR starts with chart name (e.g. [prometheus-couchdb-exporter])

@YuleZ
Copy link
Contributor Author

YuleZ commented Dec 6, 2023

@jkroepke
Copy link
Member

@YuleZ Modify the recording rules is the wrong approach here because the metrics are still collected twice.

The service monitor for the kubelet server should configured more restrictive to avoid this.

Copy link

stale bot commented Mar 13, 2024

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Any further update will cause the issue/pull request to no longer be considered stale. Thank you for your contributions.

@YuleZ YuleZ closed this Mar 21, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants