Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

statsd: aggregate metrics from a chunk in batch #146

Open
wants to merge 1 commit into
base: aiven-release-v1.32.0
Choose a base branch
from

Conversation

ezotrank
Copy link

Process all metrics in a chunk together.

This is necessary for custom Prometheus-like histogram metrics. For example, when creating an http_response_duration_seconds metric with buckets 1...100 and a flush interval in Telegraf set to 30 seconds, the old version of the code would process each sample separately. If a flush interval occurred, only the processed metrics would be flushed with the current timestamp. The next metrics would be flushed in the following interval with a different timestamp (+30s), causing issues with histogram calculation. In the current version, all samples are processed together.

Required for all PRs

resolves #

@ezotrank ezotrank self-assigned this Oct 14, 2024
Process all metrics in a chunk together.

This is necessary for custom Prometheus-like histogram metrics. For
example, when creating an `http_response_duration_seconds` metric with
buckets 1...100 and a flush interval in Telegraf set to 30 seconds,
the old version of the code would process each sample separately. If a
flush interval occurred, only the processed metrics would be flushed
with the current timestamp. The next metrics would be flushed in the
following interval with a different timestamp (+30s), causing issues
with histogram calculation. In the current version, all samples are
processed together.
@ezotrank ezotrank force-pushed the kremenev/statsd-batch-v1.32.0 branch from 88cd0d9 to 983389e Compare October 14, 2024 16:37
Base automatically changed from kremenev/telegraf-v1.32.0 to aiven-release-v1.32.0 October 15, 2024 12:37
@ezotrank ezotrank marked this pull request as ready for review October 15, 2024 12:37
@ezotrank ezotrank requested a review from RommelLayco October 16, 2024 08:14
Copy link

@RommelLayco RommelLayco left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm.
I just want someone with more go knowledge to also review before merging

@facetoe
Copy link
Member

facetoe commented Oct 17, 2024

We should really be making changes upstream and not in our fork. Leads to maintenance nightmare.

@facetoe
Copy link
Member

facetoe commented Oct 17, 2024

Ideally we could get these changes merged upstream and get our fork closer to head of upstream cause we are lagging.

Copy link
Member

@facetoe facetoe left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm going to be really annoying and request changes here as we don't really want to be adding features to our fork that are not in upstream? At least we should try upstream first.

@ezotrank
Copy link
Author

I'm going to be really annoying and request changes here as we don't really want to be adding features to our fork that are not in upstream? At least we should try upstream first.

We already have a nightmare to sync with upstream (#144) :sadcat: but I agree, that things might be good for others it should be pushed to upstream, and I'm already working on it, but it needs some time...

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants