You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I have a table with a few hundred columns and want to unpivot only 10 columns into two columns (key, value) and keep 3 other columns as "index" columns.
With the current unpivot() there doesn't appear to be an easy way to explicitly define the 10 columns that need to unpivoted and to keep the 3 columns.
unpivot() has the following parameters. My understanding is to achieve my goal, I need to pass the other 100-odd columns to the exclude parameter, which is very inconvenient.
{{ dbt_utils.unpivot(
relation=ref('table_name'),
cast_to='datatype',
exclude=[<list of columns to exclude from unpivot>],
remove=[<list of columns to remove>],
field_name=<column name for field>,
value_name=<column name for value>
) }}
It would make unpivot() easier to use if two parameters were added:
"keep" - column names to be kept unchanged.
"include" - column names to be unpivoted.
Describe alternatives you've considered
I looked at using get_filtered_columns_in_relation() to get the list of columns for exclusion, but again I need manually list columns to exclude and there does not appear to do pattern matching. In R it's easy to use starts_with() (https://tidyselect.r-lib.org/reference/starts_with.html) for example to do pattern matching of fields.
Additional context
N/A
Who will this benefit?
All DBT users.
Are you interested in contributing this feature?
No.
The text was updated successfully, but these errors were encountered:
Describe the feature
I have a table with a few hundred columns and want to unpivot only 10 columns into two columns (key, value) and keep 3 other columns as "index" columns.
With the current
unpivot()
there doesn't appear to be an easy way to explicitly define the 10 columns that need to unpivoted and to keep the 3 columns.unpivot()
has the following parameters. My understanding is to achieve my goal, I need to pass the other 100-odd columns to the exclude parameter, which is very inconvenient.It would make
unpivot()
easier to use if two parameters were added:"keep" - column names to be kept unchanged.
"include" - column names to be unpivoted.
Describe alternatives you've considered
I looked at using get_filtered_columns_in_relation() to get the list of columns for exclusion, but again I need manually list columns to exclude and there does not appear to do pattern matching. In R it's easy to use
starts_with()
(https://tidyselect.r-lib.org/reference/starts_with.html) for example to do pattern matching of fields.Additional context
N/A
Who will this benefit?
All DBT users.
Are you interested in contributing this feature?
No.
The text was updated successfully, but these errors were encountered: