Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

unpivot: add parameters for explicitly defining columns to pivot on, rather than just listing columns to exclude. #949

Open
JustGitting opened this issue Sep 12, 2024 · 0 comments
Labels
enhancement New feature or request triage

Comments

@JustGitting
Copy link

Describe the feature

I have a table with a few hundred columns and want to unpivot only 10 columns into two columns (key, value) and keep 3 other columns as "index" columns.

With the current unpivot() there doesn't appear to be an easy way to explicitly define the 10 columns that need to unpivoted and to keep the 3 columns.

unpivot() has the following parameters. My understanding is to achieve my goal, I need to pass the other 100-odd columns to the exclude parameter, which is very inconvenient.

{{ dbt_utils.unpivot(
  relation=ref('table_name'),
  cast_to='datatype',
  exclude=[<list of columns to exclude from unpivot>],
  remove=[<list of columns to remove>],
  field_name=<column name for field>,
  value_name=<column name for value>
) }}

It would make unpivot() easier to use if two parameters were added:

"keep" - column names to be kept unchanged.
"include" - column names to be unpivoted.

Describe alternatives you've considered

I looked at using get_filtered_columns_in_relation() to get the list of columns for exclusion, but again I need manually list columns to exclude and there does not appear to do pattern matching. In R it's easy to use starts_with() (https://tidyselect.r-lib.org/reference/starts_with.html) for example to do pattern matching of fields.

Additional context

N/A

Who will this benefit?

All DBT users.

Are you interested in contributing this feature?

No.

@JustGitting JustGitting added enhancement New feature or request triage labels Sep 12, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request triage
Projects
None yet
Development

No branches or pull requests

1 participant