Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enclose column name with backquote when appending a new column #859

Closed
rariyama-k opened this issue Dec 2, 2024 · 3 comments · Fixed by #861
Closed

Enclose column name with backquote when appending a new column #859

rariyama-k opened this issue Dec 2, 2024 · 3 comments · Fixed by #861
Labels
enhancement New feature or request

Comments

@rariyama-k
Copy link

Describe the feature

When you define a model like the following and execute dbt run, new columns will be added if there are schema changes.

version: 2
models:
  - name: table
    config:
      materialized: incremental
      on_schema_change: append_new_columns

Internally, the following ALTER TABLE statement is executed:

alter table `db`.`schema`.`table`
    add columns (
        new_column string
    )

At this time, if the column name contains a hyphen, it will fail with the following error:

[INVALID_IDENTIFIER] The unquoted identifier column_name is invalid and must be back quoted as: `column_name`.
It would be helpful if you could either add an option to enclose column names with backticks when executing ALTER TABLE, or modify the SQL for ALTER TABLE itself.

It would be helpful if you could either add an option to enclose column names with backticks when executing ALTER TABLE, or modify the SQL for ALTER TABLE itself.

Describe alternatives you've considered

Update alter table statement or add options whether enclosing columns with backquote or not.

Additional context

NA

Who will this benefit?

Users managing tables which has hyphen named columns will get the benefit.

Are you interested in contributing this feature?

Yes, I'll support as much as possible.

@rariyama-k rariyama-k added the enhancement New feature or request label Dec 2, 2024
@benc-db
Copy link
Collaborator

benc-db commented Dec 2, 2024

Thanks for reporting.

@benc-db
Copy link
Collaborator

benc-db commented Dec 2, 2024

I believe this is coming from spark__alter_relation_add_remove_columns macro. Could you please file a bug against dbt-spark? I'll replace in dbt-databricks, but it would be nice to make them aware of the issue.

@rariyama-k
Copy link
Author

Thank you for prompt response. I submitted the bug report in dbt-spark. Please check it when you have time.
dbt-labs/dbt-spark#1149

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants