Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

MQE: fix issues with vector/vector binary comparsion operations #10235

Draft
wants to merge 43 commits into
base: main
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
43 commits
Select commit Hold shift + click to select a range
dc874aa
Add feature toggle
charleskorn Nov 25, 2024
d9a467e
Enable upstream test cases
charleskorn Nov 26, 2024
aad0179
Remove comments about and/or/unless.
charleskorn Nov 26, 2024
37434ad
Add some test cases
charleskorn Nov 26, 2024
31c886f
Make condition clearer
charleskorn Nov 27, 2024
6ec6e0e
Add comparison operation edge cases
charleskorn Nov 27, 2024
fa12925
Add tests for comparison operators with group_left / group_right
charleskorn Nov 27, 2024
a09c1ac
Update tests to reflect https://github.com/prometheus/prometheus/issu…
charleskorn Nov 27, 2024
4965398
Expand test to confirm label handling behaviour
charleskorn Nov 27, 2024
64114f7
Rename existing operator
charleskorn Nov 27, 2024
749789b
Add structure for new operator
charleskorn Nov 27, 2024
9ce7407
Initial implementation of SeriesMetadata
charleskorn Dec 2, 2024
882b2a5
Initial implementation
charleskorn Dec 2, 2024
0ea8687
Handle case where one set of many-side series many to many output series
charleskorn Dec 3, 2024
b29c99e
Use correct labels when grouping one side series
charleskorn Dec 3, 2024
11bf6b0
Return a conflict error message if there are multiple samples at the …
charleskorn Dec 3, 2024
d48be6a
Fix handling of cases where additional labels appear on the "many" side
charleskorn Dec 3, 2024
4e1f6fa
Return a non-misleading error message when a conflict occurs on the "…
charleskorn Dec 3, 2024
1876813
Update comments
charleskorn Dec 3, 2024
d35f355
Fix regression in comparison operation output labels
charleskorn Dec 4, 2024
87bbd46
Disable one-to-one comparison operation cases that fail for known rea…
charleskorn Dec 4, 2024
a53580a
Fix linting warnings and simplify `computeOutputSeries()`
charleskorn Dec 4, 2024
331cf70
Add tests for annotations
charleskorn Dec 4, 2024
606a96d
Add tests for case where additional labels are not present on series …
charleskorn Dec 4, 2024
8c56b2d
Add series sorting test
charleskorn Dec 4, 2024
25be40f
Add provenance comment
charleskorn Dec 4, 2024
502245f
Add benchmark
charleskorn Dec 4, 2024
0b8fa34
Expand comments
charleskorn Dec 4, 2024
7e4113a
Fix typo in test names
charleskorn Dec 4, 2024
c41b575
Add test cases with label names in different orders
charleskorn Dec 4, 2024
230adfe
Add some test cases with native histograms
charleskorn Dec 4, 2024
a6d8763
Ensure buffers passed to labels.Labels.Bytes(), BytesWithLabels() and…
charleskorn Dec 4, 2024
e467023
Address PR feedback: use minimal number of points for binary operatio…
charleskorn Dec 10, 2024
aabd093
Address PR feedback: rename `latestSeries` to `latestSeriesIndex`
charleskorn Dec 10, 2024
8c8f782
Address PR feedback: add docstring for `updatePresence`
charleskorn Dec 10, 2024
7aa29ee
Address PR feedback: try to reuse slices in more cases
charleskorn Dec 10, 2024
153f66f
Run mixed metrics tests in parallel
charleskorn Dec 10, 2024
3a262bf
Add `group_left` to mixed metrics tests
charleskorn Dec 10, 2024
d2e2a1c
Address PR feedback: refactor `vectorVectorBinaryOperationEvaluator.c…
charleskorn Dec 10, 2024
2f22459
Add early filtering test case for group_right
charleskorn Dec 13, 2024
f8b79e8
Fix issue where comparison operations without the bool modifier would…
charleskorn Dec 13, 2024
cad9fa0
Avoid expensive `labels.Labels.String()` call
charleskorn Dec 13, 2024
d8b53a1
Fix issue where comparison operations between two vectors incorrectly…
charleskorn Dec 13, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 11 additions & 0 deletions cmd/mimir/config-descriptor.json
Original file line number Diff line number Diff line change
Expand Up @@ -2043,6 +2043,17 @@
"fieldFlag": "querier.mimir-query-engine.enable-histogram-quantile-function",
"fieldType": "boolean",
"fieldCategory": "experimental"
},
{
"kind": "field",
"name": "enable_one_to_many_and_many_to_one_binary_operations",
"required": false,
"desc": "Enable support for one-to-many and many-to-one binary operations (group_left/group_right) in the Mimir query engine. Only applies if the MQE is in use.",
"fieldValue": null,
"fieldDefaultValue": true,
"fieldFlag": "querier.mimir-query-engine.enable-one-to-many-and-many-to-one-binary-operations",
"fieldType": "boolean",
"fieldCategory": "experimental"
}
],
"fieldValue": null,
Expand Down
2 changes: 2 additions & 0 deletions cmd/mimir/help-all.txt.tmpl
Original file line number Diff line number Diff line change
Expand Up @@ -2105,6 +2105,8 @@ Usage of ./cmd/mimir/mimir:
[experimental] Enable support for binary logical operations in the Mimir query engine. Only applies if the MQE is in use. (default true)
-querier.mimir-query-engine.enable-histogram-quantile-function
[experimental] Enable support for the histogram_quantile function in the Mimir query engine. Only applies if the MQE is in use. (default true)
-querier.mimir-query-engine.enable-one-to-many-and-many-to-one-binary-operations
[experimental] Enable support for one-to-many and many-to-one binary operations (group_left/group_right) in the Mimir query engine. Only applies if the MQE is in use. (default true)
-querier.mimir-query-engine.enable-scalar-scalar-binary-comparison-operations
[experimental] Enable support for binary comparison operations between two scalars in the Mimir query engine. Only applies if the MQE is in use. (default true)
-querier.mimir-query-engine.enable-scalars
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -1536,6 +1536,12 @@ mimir_query_engine:
# Mimir query engine. Only applies if the MQE is in use.
# CLI flag: -querier.mimir-query-engine.enable-histogram-quantile-function
[enable_histogram_quantile_function: <boolean> | default = true]

# (experimental) Enable support for one-to-many and many-to-one binary
# operations (group_left/group_right) in the Mimir query engine. Only applies
# if the MQE is in use.
# CLI flag: -querier.mimir-query-engine.enable-one-to-many-and-many-to-one-binary-operations
[enable_one_to_many_and_many_to_one_binary_operations: <boolean> | default = true]
```

### frontend
Expand Down
3 changes: 3 additions & 0 deletions pkg/streamingpromql/benchmarks/benchmarks.go
Original file line number Diff line number Diff line change
Expand Up @@ -164,6 +164,9 @@ func TestCases(metricSizes []int) []BenchCase {
{
Expr: "nh_X / 2",
},
{
Expr: "h_X * on(l) group_left() a_X",
},
// Test the case where one side of a binary operation has many more series than the other.
{
Expr: `a_100{l=~"[13579]."} - b_100`,
Expand Down
3 changes: 3 additions & 0 deletions pkg/streamingpromql/config.go
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,7 @@ type FeatureToggles struct {
EnableScalars bool `yaml:"enable_scalars" category:"experimental"`
EnableSubqueries bool `yaml:"enable_subqueries" category:"experimental"`
EnableHistogramQuantileFunction bool `yaml:"enable_histogram_quantile_function" category:"experimental"`
EnableOneToManyAndManyToOneBinaryOperations bool `yaml:"enable_one_to_many_and_many_to_one_binary_operations" category:"experimental"`
}

// EnableAllFeatures enables all features supported by MQE, including experimental or incomplete features.
Expand All @@ -39,6 +40,7 @@ var EnableAllFeatures = FeatureToggles{
true,
true,
true,
true,
}

func (t *FeatureToggles) RegisterFlags(f *flag.FlagSet) {
Expand All @@ -50,4 +52,5 @@ func (t *FeatureToggles) RegisterFlags(f *flag.FlagSet) {
f.BoolVar(&t.EnableScalars, "querier.mimir-query-engine.enable-scalars", true, "Enable support for scalars in the Mimir query engine. Only applies if the MQE is in use.")
f.BoolVar(&t.EnableSubqueries, "querier.mimir-query-engine.enable-subqueries", true, "Enable support for subqueries in the Mimir query engine. Only applies if the MQE is in use.")
f.BoolVar(&t.EnableHistogramQuantileFunction, "querier.mimir-query-engine.enable-histogram-quantile-function", true, "Enable support for the histogram_quantile function in the Mimir query engine. Only applies if the MQE is in use.")
f.BoolVar(&t.EnableOneToManyAndManyToOneBinaryOperations, "querier.mimir-query-engine.enable-one-to-many-and-many-to-one-binary-operations", true, "Enable support for one-to-many and many-to-one binary operations (group_left/group_right) in the Mimir query engine. Only applies if the MQE is in use.")
}
82 changes: 61 additions & 21 deletions pkg/streamingpromql/engine_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -50,12 +50,10 @@ func TestUnsupportedPromQLFeatures(t *testing.T) {
// The goal of this is not to list every conceivable expression that is unsupported, but to cover all the
// different cases and make sure we produce a reasonable error message when these cases are encountered.
unsupportedExpressions := map[string]string{
"metric{} + on() group_left() other_metric{}": "binary expression with many-to-one matching",
"metric{} + on() group_right() other_metric{}": "binary expression with one-to-many matching",
"topk(5, metric{})": "'topk' aggregation with parameter",
`count_values("foo", metric{})`: "'count_values' aggregation with parameter",
"quantile_over_time(0.4, metric{}[5m])": "'quantile_over_time' function",
"quantile(0.95, metric{})": "'quantile' aggregation with parameter",
"topk(5, metric{})": "'topk' aggregation with parameter",
`count_values("foo", metric{})`: "'count_values' aggregation with parameter",
"quantile_over_time(0.4, metric{}[5m])": "'quantile_over_time' function",
"quantile(0.95, metric{})": "'quantile' aggregation with parameter",
}

for expression, expectedError := range unsupportedExpressions {
Expand Down Expand Up @@ -157,12 +155,20 @@ func TestUnsupportedPromQLFeaturesWithFeatureToggles(t *testing.T) {
requireQueryIsUnsupported(t, featureToggles, "sum_over_time(metric[1m:10s])", "subquery")
})

t.Run("classic histograms", func(t *testing.T) {
t.Run("histogram_quantile function", func(t *testing.T) {
featureToggles := EnableAllFeatures
featureToggles.EnableHistogramQuantileFunction = false

requireQueryIsUnsupported(t, featureToggles, "histogram_quantile(0.5, metric)", "'histogram_quantile' function")
})

t.Run("one-to-many and many-to-one binary operations", func(t *testing.T) {
featureToggles := EnableAllFeatures
featureToggles.EnableOneToManyAndManyToOneBinaryOperations = false

requireQueryIsUnsupported(t, featureToggles, "metric{} + on() group_left() other_metric{}", "binary expression with many-to-one matching")
requireQueryIsUnsupported(t, featureToggles, "metric{} + on() group_right() other_metric{}", "binary expression with one-to-many matching")
})
}

func requireQueryIsUnsupported(t *testing.T, toggles FeatureToggles, expression string, expectedError string) {
Expand Down Expand Up @@ -2435,6 +2441,12 @@ func TestBinaryOperationAnnotations(t *testing.T) {
testCases[name] = testCase
}

cardinalities := map[string]string{
"one-to-one": "",
"many-to-one": "group_left",
"one-to-many": "group_right",
}

for op, binop := range binaryOperations {
expressions := []string{op}

Expand All @@ -2443,14 +2455,18 @@ func TestBinaryOperationAnnotations(t *testing.T) {
}

for _, expr := range expressions {
addBinopTestCase(op, fmt.Sprintf("binary %v between two floats", expr), fmt.Sprintf(`metric{type="float"} %v ignoring(type) metric{type="float"}`, expr), "float", "float", true)
addBinopTestCase(op, fmt.Sprintf("binary %v between a float on the left side and a histogram on the right", expr), fmt.Sprintf(`metric{type="float"} %v ignoring(type) metric{type="histogram"}`, expr), "float", "histogram", binop.floatHistogramSupported)
addBinopTestCase(op, fmt.Sprintf("binary %v between a scalar on the left side and a histogram on the right", expr), fmt.Sprintf(`2 %v metric{type="histogram"}`, expr), "float", "histogram", binop.floatHistogramSupported)
addBinopTestCase(op, fmt.Sprintf("binary %v between a histogram on the left side and a float on the right", expr), fmt.Sprintf(`metric{type="histogram"} %v ignoring(type) metric{type="float"}`, expr), "histogram", "float", binop.histogramFloatSupported)
addBinopTestCase(op, fmt.Sprintf("binary %v between a histogram on the left side and a scalar on the right", expr), fmt.Sprintf(`metric{type="histogram"} %v 2`, expr), "histogram", "float", binop.histogramFloatSupported)
addBinopTestCase(op, fmt.Sprintf("binary %v between two histograms", expr), fmt.Sprintf(`metric{type="histogram"} %v ignoring(type) metric{type="histogram"}`, expr), "histogram", "histogram", binop.histogramHistogramSupported)

for cardinalityName, cardinalityModifier := range cardinalities {
addBinopTestCase(op, fmt.Sprintf("binary %v between two floats with %v matching", expr, cardinalityName), fmt.Sprintf(`metric{type="float"} %v ignoring(type) %v metric{type="float"}`, expr, cardinalityModifier), "float", "float", true)
addBinopTestCase(op, fmt.Sprintf("binary %v between a float on the left side and a histogram on the right with %v matching", expr, cardinalityName), fmt.Sprintf(`metric{type="float"} %v ignoring(type) %v metric{type="histogram"}`, expr, cardinalityModifier), "float", "histogram", binop.floatHistogramSupported)
addBinopTestCase(op, fmt.Sprintf("binary %v between a histogram on the left side and a float on the right with %v matching", expr, cardinalityName), fmt.Sprintf(`metric{type="histogram"} %v ignoring(type) %v metric{type="float"}`, expr, cardinalityModifier), "histogram", "float", binop.histogramFloatSupported)
addBinopTestCase(op, fmt.Sprintf("binary %v between two histograms with %v matching", expr, cardinalityName), fmt.Sprintf(`metric{type="histogram"} %v ignoring(type) %v metric{type="histogram"}`, expr, cardinalityModifier), "histogram", "histogram", binop.histogramHistogramSupported)
}
}
}

runAnnotationTests(t, testCases)
}

Expand Down Expand Up @@ -2649,6 +2665,8 @@ func runMixedMetricsTests(t *testing.T, expressions []string, pointsPerSeries in
}

func TestCompareVariousMixedMetricsFunctions(t *testing.T) {
t.Parallel()

labelsToUse, pointsPerSeries, seriesData := getMixedMetricsForTests(true)

// Test each label individually to catch edge cases in with single series
Expand Down Expand Up @@ -2681,6 +2699,8 @@ func TestCompareVariousMixedMetricsFunctions(t *testing.T) {
}

func TestCompareVariousMixedMetricsBinaryOperations(t *testing.T) {
t.Parallel()

labelsToUse, pointsPerSeries, seriesData := getMixedMetricsForTests(false)

// Generate combinations of 2 and 3 labels. (e.g., "a,b", "e,f", "c,d,e" etc)
Expand All @@ -2691,36 +2711,52 @@ func TestCompareVariousMixedMetricsBinaryOperations(t *testing.T) {

for _, labels := range labelCombinations {
for _, op := range []string{"+", "-", "*", "/", "and", "unless", "or"} {
binaryExpr := fmt.Sprintf(`series{label="%s"}`, labels[0])
expr := fmt.Sprintf(`series{label="%s"}`, labels[0])
for _, label := range labels[1:] {
binaryExpr += fmt.Sprintf(` %s series{label="%s"}`, op, label)
expr += fmt.Sprintf(` %s series{label="%s"}`, op, label)
}
expressions = append(expressions, binaryExpr)
expressions = append(expressions, expr)

// Same thing again, this time with grouping.
binaryExpr = fmt.Sprintf(`series{label="%s"}`, labels[0])
expr = fmt.Sprintf(`series{label="%s"}`, labels[0])
for i, label := range labels[1:] {
binaryExpr += fmt.Sprintf(` %s ignoring (label, group) `, op)
expr += fmt.Sprintf(` %s ignoring (label, group) `, op)

if i == 0 && len(labels) > 2 {
binaryExpr += "("
expr += "("
}

binaryExpr += fmt.Sprintf(`{label="%s"}`, label)
expr += fmt.Sprintf(`{label="%s"}`, label)
}

if len(labels) > 2 {
binaryExpr += ")"
expr += ")"
}
expressions = append(expressions, expr)
}

// Similar thing again, this time with group_left
expr := fmt.Sprintf(`series{label="%s"}`, labels[0])
for i, label := range labels[1:] {
expr += ` * on(group) group_left(label) `

if i == 0 && len(labels) > 2 {
expr += "("
}

expressions = append(expressions, binaryExpr)
expr += fmt.Sprintf(`{label="%s"}`, label)
}
if len(labels) > 2 {
expr += ")"
}
expressions = append(expressions, expr)
}

runMixedMetricsTests(t, expressions, pointsPerSeries, seriesData, false)
}

func TestCompareVariousMixedMetricsAggregations(t *testing.T) {
t.Parallel()

labelsToUse, pointsPerSeries, seriesData := getMixedMetricsForTests(true)

// Test each label individually to catch edge cases in with single series
Expand Down Expand Up @@ -2749,6 +2785,8 @@ func TestCompareVariousMixedMetricsAggregations(t *testing.T) {
}

func TestCompareVariousMixedMetricsVectorSelectors(t *testing.T) {
t.Parallel()

labelsToUse, pointsPerSeries, seriesData := getMixedMetricsForTests(true)

// Test each label individually to catch edge cases in with single series
Expand All @@ -2774,6 +2812,8 @@ func TestCompareVariousMixedMetricsVectorSelectors(t *testing.T) {
}

func TestCompareVariousMixedMetricsComparisonOps(t *testing.T) {
t.Parallel()

labelsToUse, pointsPerSeries, seriesData := getMixedMetricsForTests(true)

// Test each label individually to catch edge cases in with single series
Expand Down
6 changes: 4 additions & 2 deletions pkg/streamingpromql/operators/aggregations/aggregation.go
Original file line number Diff line number Diff line change
Expand Up @@ -212,7 +212,8 @@ func (a *Aggregation) groupingWithoutLabelsSeriesToGroupFuncs() (seriesToGroupLa
// Why 1024 bytes? It's what labels.Labels.String() uses as a buffer size, so we use that as a sensible starting point too.
b := make([]byte, 0, 1024)
bytesFunc := func(l labels.Labels) []byte {
return l.BytesWithoutLabels(b, a.Grouping...) // NewAggregation will add __name__ to Grouping for 'without' aggregations, so no need to add it here.
b = l.BytesWithoutLabels(b, a.Grouping...) // NewAggregation will add __name__ to Grouping for 'without' aggregations, so no need to add it here.
return b
}

lb := labels.NewBuilder(labels.EmptyLabels())
Expand All @@ -231,7 +232,8 @@ func (a *Aggregation) groupingByLabelsSeriesToGroupFuncs() (seriesToGroupLabelsB
// Why 1024 bytes? It's what labels.Labels.String() uses as a buffer size, so we use that as a sensible starting point too.
b := make([]byte, 0, 1024)
bytesFunc := func(l labels.Labels) []byte {
return l.BytesWithLabels(b, a.Grouping...)
b = l.BytesWithLabels(b, a.Grouping...)
return b
}

lb := labels.NewBuilder(labels.EmptyLabels())
Expand Down
Loading
Loading