expect_column_pair_values_a_to_be_greater_than_b fail to show unexpected rows

[VytSep]

Describe the bug
The expectation expect_column_pair_values_a_to_be_greater_than_b is not generating the unexpected rows

To Reproduce
The expectation is configured this way:
COMPLETE={"result_format": "COMPLETE", "include_unexpected_rows": True}
expectation = gx.expectations.ExpectColumnPairValuesAToBeGreaterThanB(
column_A="date_a",
column_B="date_b",
or_equal=True,
result_format=COMPLETE,
)

The checkpoint is configured this way:
checkpoint_name = "my_checkpoint"
checkpoint = gx.Checkpoint(
name=checkpoint_name,
validation_definitions=validation_definitions,
result_format={"result_format": "COMPLETE", "include_unexpected_rows": True},
)

Expected behavior
Have in the validation result the unexpected rows for this expectation

Environment (please complete the following information):

• Great Expectations Version: [1.0.0]
• Cloud environment: [Databricks]

Additional context
For other expectations like expect_column_values_to_be_in_set I do have the unexpected rows generated

[adeola-ak]

hi there - just want to let you know i am looking into this. to my knowledge, both should be returning unexpected rows. Please stand by while I continue to investigate

[adeola-ak]

following up here, what are you getting it back? do you see the key returning the object for unexpected rows like you do for expect_column_values_to_be_in_set at all?

[VytSep]

I do have key returning for the expect_column_values_to_be_in_set and also for the expect_column_pair_values_a_to_be_greater_than_b

[adeola-ak]

so you have the unexpected values key being returned for both but for expect_column_pair_values_a_to_be_greater_than_b the key is just empty? can you share a ss?

[VytSep]

Yes, I do have the keys for both expectations. What do you mean for ss?

[adeola-ak]

screenshot i'm sorry

[VytSep]

Sorry, I don't have permission to share screenshots

[adeola-ak]

understood no problem! but again, is the unexpected_rows key empty or returning null or something else?

if you edit your project to only have the expect_column_pair_values_a_to_be_greater_than_b expectation, and run print(validation_results["results"][0]["result"]["unexpected_rows"]) what do you see? As i'm seeing None across a few projects, I do need to clarify again what is expected with such an expecation that is evaluated for a pair of columns. I will get a concrete answer to you by today

[VytSep]

[VytSep]

Hi team, any news on that subject?

[adeola-ak]

Hi @VytSep, thank you for your patience. This issue has been confirmed as a bug. Unfortunately, I don't have a specific timeline for when it will be addressed, but I plan to follow up with the team tomorrow and should have an update then.

[adeola-ak]

@VytSep did this work for you in the past? or prior to 1.0?

[VytSep]

Hi @adeola-ak , no i did not. I have the same issue also with the 0.18.15 version.

[adeola-ak]

hi @VytSep sorry that I misspoke earlier this month. After further review, though unexpected rows works for ColumnMapExpectations, this behavior is expected for all ColumnPairMapExpectations. I will update our documentation to reflect this. For now, having this functionality would be considered a feature request, which is not prioritized at the moment. Thank you for your patience, and I apologize if this isn't the outcome you were hoping for.