pod stats get duplicated in some situations

[puckpuck]

The agent seems to return 2 events for the same pod on some collection cycles (not all). With closer examination only the metrics.uptime, and sometimes the network metrics are different but all other stats are the same. There should only be a single event per pod per collection cycle returned.

This may be due to the kubelet returning the same pod multiple times when pulled for stats.
This query on the Honeycomb demo team shows the issue: https://ui.honeycomb.io/demo/environments/ms-demo/datasets/kubernetes-metrics/result/ywxDQmCmhMP
https://houndsh.slack.com/archives/CDE86Q55X/p1660658679350499

[kentquirk]

@puckpuck This turned out to be ... complicated. We're currently putting it in the icebox pending capacity to deal with it properly.

[MikeGoldsmith]

We are trimming our OSS backlog. We will be closing this issue as it is a low priority for us to fix. It is unlikely that we'll ever get to it, and so we'd like to set expectations accordingly.

If this issue is important to you, please feel free to ping here and we can discuss/re-open.