[kube-prometheus-stack] fix(prometheus): rm image!="" from k8s.rules

[jjangga0214]

What this PR does / why we need it

Which issue this PR fixes

"NO DATA" issues for pod CPU.

BEFORE

AFTER(PR)

Special notes for your reviewer

"NO DATA" issues are still happening to others (not cpu usage) metrics (regardless of this PR).
They should be fixed as well, but how?

Checklist

• DCO signed
• Chart Version bumped
• Title of the PR starts with chart name (e.g. [prometheus-couchdb-exporter])

[jkroepke]

@jjangga0214 which CRI are you using? Because omit image!="" would result that CPU usage is twice high.

[jjangga0214]

@jkroepke The screenshot was captured against the cluster on "Docker for Desktop" on macOS.
So the CRI is Docker. Maybe this "No Data" issue only happens on "Docker for Desktop"?

[jkroepke]

I remember correctly, if CRI is docker. Could be verifiy with minikube where CRI is default to docker and can be configured to containerd.

[jjangga0214]

@jkroepke I installed the chart(not modified by this PR) into pure k8s (provisioned by kubespray), but the same issue happens. Its CRI is configured as containerd.

[QuentinBisson]

@jjangga0214 could you provide that upstream as described here https://github.com/prometheus-community/helm-charts/blob/30ba9d6f0815cd100bfb73d209ecbffa158bda5f/charts/kube-prometheus-stack/templates/prometheus/rules-1.14/k8s.rules.yaml#L2C1-L2C1 ?

[jjangga0214]

@QuentinBisson Hi! Thanks for the reply.
I don't understand what you are asking for.
Would you please reword more concretely?
Do you mean the content of PrometheusRule CRD from my cluster?

By the way, I just installed v51.10.0 chart, and the issue happens as I described.
I didn't modify the chart.
Then isn't the version itself enough information to provide?

[QuentinBisson]

@jjangga0214 what I mean is that you should push the change here https://raw.githubusercontent.com/prometheus-operator/kube-prometheus/main/manifests/kubernetesControlPlane-prometheusRule.yaml instead and then come here to sync the rules because we are only consuming the rules from another project

[deefdragon]

Ive done some thinking and I think the solution hinted at in 3972 (to create a cadvisor chart that gets deployed with the stack) would be a more appropriate path to go down. To deploy a dedicated cadvisor monitor as part of the stack, as opposed to attempting to wrestle with missing metrics would solve the issue for the existing grafana graphs, but also for the Lens K8S UI issues.

I feel having one solution to solve both (and likely more) problems at once is likely better than playing wackamole for each now-broken graph & metric.

Unfortunately, I have no idea what the workload on patching that in would be.

[jkroepke]

Hi,

I'm closing the PR now. I understand that the error may exists, however as @QuentinBisson mention, the kube-prometheus-stack rules are generated from kube-prometheus and kubernetes-mixin

A manual overwrite would result in a revert. on the next update.

If you still have the issues, consider to create an issue for this.