Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Clarify internal usage of empty label values by federation #2530

Open
wants to merge 2 commits into
base: main
Choose a base branch
from
Open

Clarify internal usage of empty label values by federation #2530

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
5ca6b27
Clarify internal usage of empty label values by federation
alexgreenbank Oct 8, 2024
35b3b36
add to remote write 2.0 spec too
alexgreenbank Oct 8, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions content/docs/specs/remote_write_spec.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -128,6 +128,8 @@ Label names beginning with "__" are RESERVED for system usage and SHOULD NOT be

Remote write Receivers MAY ingest valid samples within a write request that otherwise contains invalid samples. Receivers MUST return a HTTP 400 status code ("Bad Request") for write requests that contain any invalid samples. Receivers SHOULD provide a human readable error message in the response body. Senders MUST NOT try and interpret the error message, and SHOULD log it as is.

Whilst [Federation](https://prometheus.io/docs/prometheus/latest/federation/) relies upon an `instance` label with an empty value this is strictly an internal use case. Everything else MUST adhere to the `MUST NOT contain any empty label names or values` requirement above.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤔 I'm wondering if the federation use case would ever lead to the necessity to send an empty label value via remote write.

In different news, I think the most common case of utilizing an empty instance label is actually the Pushgateway and not federation. (The Pushgateway even adds an empty instance label implicitly, see last paragraph of this doc section.)

But again, this is only relevant for exposition and subsequent scrape, not for remote write. The samples ingested via federation or from a Pushgateway won't have any empty labels anymore when sent via remote write.

So maybe let's take it from a different perspective:

Suggested change
Whilst [Federation](https://prometheus.io/docs/prometheus/latest/federation/) relies upon an `instance` label with an empty value this is strictly an internal use case. Everything else MUST adhere to the `MUST NOT contain any empty label names or values` requirement above.
The existing use cases for empty label names (mostly when scraping a [federation endpoint](https://prometheus.io/docs/prometheus/latest/federation/) or the [Pushgateway](https://github.com/prometheus/pushgateway?tab=readme-ov-file#about-the-job-and-instance-labels) with the `honor_labels: true` setting) are irrelevant for remote write and thus not in conflict with the “MUST NOT contain any empty label names or values” requirement above.


### Ordering
Prometheus Remote Write compatible senders MUST send samples for any given series in timestamp order. Prometheus Remote Write compatible Senders MAY send multiple requests for different series in parallel.

Expand Down
2 changes: 2 additions & 0 deletions content/docs/specs/remote_write_spec_2_0.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -399,6 +399,8 @@ Label names beginning with "__" are RESERVED for system usage and SHOULD NOT be

Receivers also MAY impose limits on the number and length of labels, but this is receiver-specific and is out of the scope of this document.

Whilst [Federation](https://prometheus.io/docs/prometheus/latest/federation/) relies upon an `instance` label with an empty value this is strictly an internal use case. Everything else MUST adhere to the `MUST NOT contain any empty label names or values` requirement above.

#### Samples and Histogram Samples

<!---
Expand Down