Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix flaky test related to tracking events #3604

Merged
merged 4 commits into from
Jul 26, 2021
Merged

Conversation

kwigley
Copy link
Contributor

@kwigley kwigley commented Jul 21, 2021

resolves #3561

Description

  • skips comment out tests related to event tracking, these are testing non-deterministic behavior
  • allow null users when calling tracking event code, skip sending events downstream if this is the case

Checklist

  • I have signed the CLA
  • I have run this code in development and it appears to resolve the stated issue
  • This PR includes tests, or tests are not required/relevant for this PR
  • I have updated the CHANGELOG.md and added information about my change to the "dbt next" section.

@cla-bot cla-bot bot added the cla:yes label Jul 21, 2021
@kwigley kwigley force-pushed the fix/flaky-tracking-tests branch from cdee6ff to ba13465 Compare July 21, 2021 04:21
@kwigley kwigley temporarily deployed to Bigquery July 21, 2021 04:23 Inactive
@kwigley kwigley temporarily deployed to Bigquery July 21, 2021 04:23 Inactive
@kwigley kwigley temporarily deployed to Snowflake July 21, 2021 04:23 Inactive
@kwigley kwigley temporarily deployed to Snowflake July 21, 2021 04:23 Inactive
@kwigley kwigley temporarily deployed to Redshift July 21, 2021 04:23 Inactive
@kwigley kwigley temporarily deployed to Redshift July 21, 2021 04:23 Inactive
@kwigley kwigley temporarily deployed to Bigquery July 21, 2021 13:17 Inactive
@kwigley kwigley temporarily deployed to Bigquery July 21, 2021 13:17 Inactive
@kwigley kwigley temporarily deployed to Redshift July 21, 2021 13:17 Inactive
@kwigley kwigley temporarily deployed to Redshift July 21, 2021 13:17 Inactive
@kwigley kwigley temporarily deployed to Snowflake July 21, 2021 13:17 Inactive
@kwigley kwigley temporarily deployed to Snowflake July 21, 2021 13:17 Inactive
@@ -9,6 +10,7 @@
import dbt.tracking
import dbt.utils

pytestmark = pytest.mark.skip("Experimental parser generates events non-deterministically, skipping.")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this skip all the tests in this file? I don't see pytestmark being used anywhere so I'm not sure how that works

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That is correct, it marks all tests in the module as skipped!

https://docs.pytest.org/en/stable/reference.html#globalvar-pytestmark

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@nathaniel-may nathaniel-may mentioned this pull request Jul 22, 2021
4 tasks
@kwigley kwigley temporarily deployed to Redshift July 26, 2021 15:17 Inactive
@kwigley kwigley temporarily deployed to Redshift July 26, 2021 15:17 Inactive
@kwigley kwigley temporarily deployed to Bigquery July 26, 2021 15:17 Inactive
@kwigley kwigley temporarily deployed to Bigquery July 26, 2021 15:17 Inactive
@kwigley kwigley temporarily deployed to Snowflake July 26, 2021 15:18 Inactive
@kwigley kwigley temporarily deployed to Snowflake July 26, 2021 15:18 Inactive
@kwigley kwigley force-pushed the fix/flaky-tracking-tests branch from 3311ef3 to 94c3fa3 Compare July 26, 2021 15:20
@kwigley kwigley temporarily deployed to Bigquery July 26, 2021 15:20 Inactive
@kwigley kwigley temporarily deployed to Bigquery July 26, 2021 19:51 Inactive
@kwigley kwigley temporarily deployed to Snowflake July 26, 2021 19:51 Inactive
@kwigley kwigley temporarily deployed to Snowflake July 26, 2021 19:51 Inactive
@kwigley kwigley temporarily deployed to Postgres July 26, 2021 19:51 Inactive
@kwigley kwigley temporarily deployed to Redshift July 26, 2021 19:51 Inactive
@kwigley kwigley temporarily deployed to Redshift July 26, 2021 19:51 Inactive
@kwigley kwigley temporarily deployed to Snowflake July 26, 2021 19:51 Inactive
@kwigley kwigley temporarily deployed to Snowflake July 26, 2021 19:51 Inactive
@kwigley kwigley force-pushed the fix/flaky-tracking-tests branch from 15261bb to 19150ea Compare July 26, 2021 20:19
@kwigley kwigley temporarily deployed to Bigquery July 26, 2021 20:19 Inactive
@kwigley kwigley temporarily deployed to Bigquery July 26, 2021 20:19 Inactive
@kwigley kwigley temporarily deployed to Postgres July 26, 2021 20:19 Inactive
@kwigley kwigley temporarily deployed to Snowflake July 26, 2021 20:19 Inactive
@kwigley kwigley temporarily deployed to Postgres July 26, 2021 20:19 Inactive
@kwigley kwigley temporarily deployed to Snowflake July 26, 2021 20:19 Inactive
@kwigley kwigley temporarily deployed to Redshift July 26, 2021 20:19 Inactive
@kwigley kwigley temporarily deployed to Redshift July 26, 2021 20:19 Inactive
@kwigley kwigley temporarily deployed to Redshift July 26, 2021 20:19 Inactive
@kwigley kwigley temporarily deployed to Redshift July 26, 2021 20:19 Inactive
@kwigley kwigley temporarily deployed to Bigquery July 26, 2021 20:19 Inactive
@kwigley kwigley temporarily deployed to Bigquery July 26, 2021 20:19 Inactive
@kwigley kwigley temporarily deployed to Snowflake July 26, 2021 20:19 Inactive
@kwigley kwigley temporarily deployed to Snowflake July 26, 2021 20:19 Inactive
@kwigley
Copy link
Contributor Author

kwigley commented Jul 26, 2021

Created #3631 to address root cause.

@kwigley kwigley merged commit 9c58f34 into develop Jul 26, 2021
@kwigley kwigley deleted the fix/flaky-tracking-tests branch July 26, 2021 20:55
leahwicz pushed a commit that referenced this pull request Jul 29, 2021
* skip all tracking event testing

* Turn off tracking in tests that hits model parsing code path
fix other random test that fails because global tracking.current_user exists but is null

* pytest did not respect skip mark

* fix gh actions
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Fix flaky tracking integration tests
3 participants