-
Notifications
You must be signed in to change notification settings - Fork 1.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add tags to SavedQueries #10987
base: main
Are you sure you want to change the base?
Add tags to SavedQueries #10987
Conversation
Thank you for your pull request! We could not find a changelog entry for this change. For details on how to document a change, see the contributing guide. |
150228b
to
f0eaad0
Compare
f0eaad0
to
2d39824
Compare
Resolves [#369](#369) ### Description Addresses internal Linear issue SL-2896. Users currently can execute parts of their DAG conditionally based on tags added to individual nodes (see [documentation](https://docs.getdbt.com/reference/resource-configs/tags)). This brings SavedQueries in line with other similar nodes and allows the use of tags as described in that documentation. (See the added tests for examples.) It does not add any hierarchical behaviors for these tags. A related [PR #10987](dbt-labs/dbt-core#10987) is in progress in dbt-core. ### Checklist - [x] I have read [the contributing guide](https://github.com/dbt-labs/dbt-semantic-interfaces/blob/main/CONTRIBUTING.md) and understand what's expected of me - [x] I have signed the [CLA](https://docs.getdbt.com/docs/contributor-license-agreements) - [x] This PR includes tests, or tests are not required/relevant for this PR - [x] I have run `changie new` to [create a changelog entry](https://github.com/dbt-labs/dbt-semantic-interfaces/blob/main/CONTRIBUTING.md#adding-a-changelog-entry)
2d39824
to
56d3899
Compare
54b5b3c
to
f2b8786
Compare
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## main #10987 +/- ##
==========================================
+ Coverage 88.85% 88.87% +0.01%
==========================================
Files 187 187
Lines 24001 24018 +17
==========================================
+ Hits 21327 21345 +18
+ Misses 2674 2673 -1
Flags with carried forward coverage won't be shown. Click here to find out more.
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM! Might be worth having folks on metadata side confirm the schema change is safe for them.
@@ -55,7 +55,8 @@ | |||
"success", | |||
"error", | |||
"skipped", | |||
"partial success" | |||
"partial success", | |||
"no-op" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I thought this had already been merged as I see it here: https://schemas.getdbt.com/dbt/run-results/v6.json
Resolves #11155
Problem
Folks would like to be able to execute exports based on tags. We can't easily support individual exports in the dbt dag right now because the exports aren't proper nodes in the dbt graph, but we can provide similar functionality by allowing them to tag the saved queries that declare those exports.
This provides several types of related functionality -
tags: ["my_tag_1", "my_tag_2"]
tags: "my_tag"
Solution
To do this, updates were made to the schemas and pydantic class for saved queries in dsi in a preceding PR. In this PR, we update how the saved queries are parsed in dbt-core (for the manifest, not the semantic-manifest). We add a little special logic to handle the case where the tags argument is just a single string and to sort and de-duplicate tags because that's just nicer.
Manual Testing
To test this locally, I ran
make dev
and then used the local version of dbt-core to parse a test project we have.I added this to the dbt_project.yml:
(only the tags lines were new)
And I added this line to a yml file's saved query :
tags: ['tag_b', 'tag_d', 'tag_a']
I ran
~/git/dbt-core/venv/bin/dbt parse
(i had some weirdness when i didn't point at my venv), and checked the manifest file and semantic manifest. they both had sections like this for the saved query:"tags": ["tag_a", "tag_b", "tag_config_1", "tag_d"]}]}
I tried it again with lists of tags in my saved queries structured like
resulting in
"tags": ["tag_2", "tag_A", "tag_config_1"]}]}
and
resulting in
"tags": ["tag_A", "tag_config_1"]}]}
Running with these tags selected
The difference in dbt_core is actually minimal.
If I run this with a selector on tags that doesn't exist, I see:
Whereas with a real tag, we build as expected:
Checklist