-
Notifications
You must be signed in to change notification settings - Fork 124
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
update external table columns #252
Merged
dataders
merged 26 commits into
dbt-labs:main
from
thomas-vl:feature/update-table-columns
Apr 11, 2024
Merged
Changes from 25 commits
Commits
Show all changes
26 commits
Select commit
Hold shift + click to select a range
1f10cc7
update external table columns
thomas-vl 5b8f60e
undo quoting
thomas-vl 3edc1ca
undo newline
thomas-vl 1f79b82
undo comma
thomas-vl c3e6c6e
add newline
thomas-vl 49c0d2e
undo newline
thomas-vl 48e3f7e
fix
thomas-vl 3d72989
fix quoting
thomas-vl 74699c5
fix
thomas-vl f3dcd89
fix
thomas-vl 550efe4
add description
thomas-vl 81830d9
add newlines
thomas-vl 2222cdb
test if this fixes the CI
thomas-vl d4a6f1b
test
thomas-vl 49eb939
test CI
thomas-vl 9aeca26
Put back changes
thomas-vl c5746f4
test
thomas-vl ce0059b
fix return statement
thomas-vl 06d47c6
test
thomas-vl 8197652
Add trigger CI job
thomas-vl f8873b9
test
thomas-vl 1046467
Add comment
thomas-vl c9281e9
Core version could be out of sync with the adapter specific version
thomas-vl 1a13283
Put protobuf back
thomas-vl c8c471d
Merge branch 'dbt-labs:main' into feature/update-table-columns
thomas-vl 2bd4a35
Merge branch 'main' into feature/update-table-columns
dataders File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,7 @@ | ||
{% macro update_external_table_columns(source_node) %} | ||
{{ return(adapter.dispatch('update_external_table_columns', 'dbt_external_tables')(source_node)) }} | ||
{% endmacro %} | ||
|
||
{% macro default__update_external_table_columns(source_node) %} | ||
|
||
{% endmacro %} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,5 @@ | ||
{% macro bigquery__update_external_table_columns(source_node) %} | ||
{%- set columns = source_node.columns -%} | ||
{%- set relation = source(source_node.source_name, source_node.name) -%} | ||
{%- do adapter.update_columns(relation, columns) -%} | ||
{% endmacro %} |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
update_external_table_columns
returns an empty string (as is the case with thedefault__
version) then it is a no-op?There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, it just updates the schema with operations that can only be done via the API, see:
https://github.com/dbt-labs/dbt-bigquery/blob/6c0afe4cfb69761dada5d16150fe632b8f72bf39/dbt/adapters/bigquery/impl.py#L609
It adds descriptions and policyTags effectively creating the same behaviour as when you create a normal model.
So I added the default__update_external_table_columns because I thought that this was how you should implement a macro that is BigQuery specific.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
perhaps i was looking for something more conventional (for this repo/ dbt macros) like