Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Tidy-First]: Fix timings object for hooks and macros, and make types of timings explicit #10882

Merged
merged 6 commits into from
Oct 18, 2024

Conversation

aranke
Copy link
Member

@aranke aranke commented Oct 18, 2024

Disclaimer

test_use_custom_microbatch_strategy_env_var_true_invalid_incremental_strategy is failing across all PRs, hence the test failures

Problem

  1. The timings object accepts any arbitrary keys, scope this down to just compile, execute, other`
  2. hook and macro nodes should report compile and execute timings independently, but they do so combined right now.

Solution

  1. Change name in timings to be a Literal
  2. Use collect_timing_info instead of creating a TimingInfo object directly

Checklist

  • I have read the contributing guide and understand what's expected of me.
  • I have run this code in development, and it appears to resolve the stated issue.
  • This PR includes tests, or tests are not required or relevant for this PR.
  • This PR has no interface changes (e.g., macros, CLI, logs, JSON artifacts, config files, adapter interface, etc.) or this PR has already received feedback and approval from Product or DX.
  • This PR includes type annotations for new and modified functions.

@cla-bot cla-bot bot added the cla:yes label Oct 18, 2024
Copy link
Contributor

Thank you for your pull request! We could not find a changelog entry for this change. For details on how to document a change, see the contributing guide.

@aranke aranke added artifact_minor_upgrade To bypass the CI check by confirming that the change is not breaking Skip Changelog Skips GHA to check for changelog file labels Oct 18, 2024
Copy link

codecov bot commented Oct 18, 2024

Codecov Report

Attention: Patch coverage is 87.09677% with 4 lines in your changes missing coverage. Please review.

Project coverage is 89.15%. Comparing base (8be0635) to head (e8ff756).
Report is 2 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main   #10882      +/-   ##
==========================================
- Coverage   89.20%   89.15%   -0.06%     
==========================================
  Files         183      183              
  Lines       23438    23443       +5     
==========================================
- Hits        20909    20901       -8     
- Misses       2529     2542      +13     
Flag Coverage Δ
integration 86.43% <87.09%> (-0.13%) ⬇️
unit 62.12% <16.12%> (-0.02%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Components Coverage Δ
Unit Tests 62.12% <16.12%> (-0.02%) ⬇️
Integration Tests 86.43% <87.09%> (-0.13%) ⬇️

@aranke aranke marked this pull request as ready for review October 18, 2024 16:26
@aranke aranke requested review from a team as code owners October 18, 2024 16:26
@aranke aranke requested review from jcserv and removed request for a team October 18, 2024 16:26
@@ -164,7 +164,7 @@ def test_use_custom_microbatch_strategy_env_var_true_invalid_incremental_strateg
):
# Initial run
with patch_microbatch_end_time("2020-01-03 13:57:00"):
run_dbt(["run"])
run_dbt(["run"], expect_pass=False)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This change is because the new dbt-adapters change altered the behavior here

Copy link
Contributor

@ChenyuLInx ChenyuLInx left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@peterallenwebb peterallenwebb merged commit ba6c7ba into main Oct 18, 2024
53 of 56 checks passed
@peterallenwebb peterallenwebb deleted the tidy_first_timings branch October 18, 2024 21:29
Copy link
Contributor

The backport to 1.9.latest failed:

The process '/usr/bin/git' failed with exit code 128

To backport manually, run these commands in your terminal:

# Fetch latest updates from GitHub
git fetch
# Create a new working tree
git worktree add .worktrees/backport-1.9.latest 1.9.latest
# Navigate to the new working tree
cd .worktrees/backport-1.9.latest
# Create a new branch
git switch --create backport-10882-to-1.9.latest
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x --mainline 1 ba6c7baf1d66d91fe7050e351e408e2a1186eee7
# Push it to GitHub
git push --set-upstream origin backport-10882-to-1.9.latest
# Go back to the original working tree
cd ../..
# Delete the working tree
git worktree remove .worktrees/backport-1.9.latest

Then, create a pull request where the base branch is 1.9.latest and the compare/head branch is backport-10882-to-1.9.latest.

QMalcolm pushed a commit that referenced this pull request Oct 24, 2024
…es of timings explicit (#10882)

* [Tidy-First]: Fix `timings` object for hooks and macros, and make types of timings explicit

* cast literal to str

* change test

* change jsonschema to enum

* Discard changes to schemas/dbt/manifest/v12.json

* nits

---------

Co-authored-by: Chenyu Li <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
artifact_minor_upgrade To bypass the CI check by confirming that the change is not breaking backport 1.9.latest cla:yes Skip Changelog Skips GHA to check for changelog file
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants