Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] Missing container metrics with multiple containers in a pod #274

Closed
oGi4i opened this issue Sep 22, 2024 · 1 comment
Closed

[BUG] Missing container metrics with multiple containers in a pod #274

oGi4i opened this issue Sep 22, 2024 · 1 comment
Assignees
Labels
bug Something isn't working stale

Comments

@oGi4i
Copy link

oGi4i commented Sep 22, 2024

Describe the bug
If a pod has more than 1 container than there is only 1 metric reported

To Reproduce
Steps to reproduce the behavior:

  1. Deploy a pod with multiple containers and enabled version check for all of them
  2. Check Prometheus with the following Query:
version_checker_is_latest_version{exported_namespace="<your_namespace>", exported_pod="<your_pod_name>"}
  1. The actual metrics are:
version_checker_is_latest_version{container="version-checker", container_type="container", current_version="<image_2_version>", endpoint="web", exported_container="<container_name_2>", exported_namespace="<your_namespace>", exported_pod="<your_pod_name>", image="<image_2>", instance="10.255.254.150:8080", job="version-checker", latest_version="<image_2_latest_version>", namespace="version-checker", pod="version-checker-6dd5b85f7d-4s86k", service="version-checker"} = 1
  1. The expected metrics should be:
version_checker_is_latest_version{container="version-checker", container_type="container", current_version="<image_1_version>", endpoint="web", exported_container="<container_name_1>", exported_namespace="<your_namespace>", exported_pod="<your_pod_name>", image="<image_1>", instance="10.255.254.150:8080", job="version-checker", latest_version="<image_1_latest_version>", namespace="version-checker", pod="version-checker-6dd5b85f7d-4s86k", service="version-checker"} = 1
version_checker_is_latest_version{container="version-checker", container_type="container", current_version="<image_2_version>", endpoint="web", exported_container="<container_name_2>", exported_namespace="<your_namespace>", exported_pod="<your_pod_name>", image="<image_2>", instance="10.255.254.150:8080", job="version-checker", latest_version="<image_2_latest_version>", namespace="version-checker", pod="version-checker-6dd5b85f7d-4s86k", service="version-checker"} = 1

Expected behavior
1 metric is reported per container

Environment (please complete the following information):

  • Version-Checker: v0.8.0+
  • Kubernetes: v1.27.16
  • Grafana: 11.2.0
  • Prometheus: 2.54.1

Additional context
The root cause of the bug is here - using partial labels to delete metrics, which only contain namespace and pod. This actually deletes all the metrics for a pod when really we want to delete only the metric for the current container that is being checked.
You can easily add a container label to this list and it should fix the issue.

@oGi4i oGi4i added the bug Something isn't working label Sep 22, 2024
Copy link
Contributor

This issue has been automatically marked as stale because it has not had
any activity in the last 90 days. It will be closed if no further activity
occurs. Thank you for your contributions.

@github-actions github-actions bot added the stale label Dec 22, 2024
@github-actions github-actions bot closed this as not planned Won't fix, can't repro, duplicate, stale Dec 29, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working stale
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants