Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug] --empty doesn't work when using get_columns_in_relation #1033

Closed
2 tasks done
georgea-tk opened this issue May 13, 2024 · 4 comments · Fixed by #1100 or #1104
Closed
2 tasks done

[Bug] --empty doesn't work when using get_columns_in_relation #1033

georgea-tk opened this issue May 13, 2024 · 4 comments · Fixed by #1100 or #1104
Assignees
Labels
backport 1.8.latest bug Something isn't working empty Issues related to the --empty CLI flag user docs [docs.getdbt.com] Needs better documentation

Comments

@georgea-tk
Copy link

georgea-tk commented May 13, 2024

Is this a new bug in dbt-snowflake?

  • I believe this is a new bug in dbt-snowflake
  • I have searched the existing issues, and I could not find an existing issue for this bug

Current Behavior

The --empty flag changes the relation name to be something like this:
(select * from DATABASE_NAME.SCHEMA_NAME.TABLE_NAME where false limit 0)
This becomes an issue when the adapter get_columns_in_relation function is called as it tries to do something like this, which is invalid snowflake sql:
describe table (select * from DATABASE_NAME.SCHEMA_NAME.TABLE_NAME where false limit 0)

This means we get the following sql errors when using functions like dbt_utils.star:
Syntax error: unexpected '('. (line 1)

Expected Behavior

anywhere where the {{ relation }} is used after a describe statement, this should be handled differently for the --empty flag to not be the subquery, but instead use the default original relation.
Like here:
https://github.com/dbt-labs/dbt-snowflake/blob/main/dbt/include/snowflake/macros/adapters.sql#L30

Steps To Reproduce

  1. dbt-snowflake 1.8
  2. running with the --empty command
  3. on any model which includes the {{ dbt_utils.star() }} macro

Relevant log output

No response

Environment

- OS: Linux amd64
- Python: 3.10
- dbt-core: 1.8.0
- dbt-snowflake: 1.8.0

Additional Context

No response

@georgea-tk georgea-tk added bug Something isn't working triage labels May 13, 2024
@jtcohen6
Copy link
Contributor

Thanks for opening @georgea-tk. I agree this is something we need to fix to fully support --empty runs.

An immediate fix I can think of is updating snowflake__get_columns_in_relation from {{ relation }} to {{ relation.render() }}, which will bypass the conditional logic here.

{% macro snowflake__get_columns_in_relation(relation) -%}
  {%- set sql -%}
    describe table {{ relation.render() }}
  {%- endset -%}
  ...
{% endmacro %}

My concern with such a fix is that we'll need to make it in a variety of places, anywhere that a ref() could conceivably be passed. I think get_columns_in_relation is one of the most common, more than drop or create_view_as, but we should aim for consistency.

@kokorin
Copy link

kokorin commented May 30, 2024

Not sure if it will be useful, but we have the same bug with relation created using DBT's source function

@FishtownBuildBot
Copy link
Collaborator

Opened a new issue in dbt-labs/docs.getdbt.com: dbt-labs/docs.getdbt.com#5732

@justin-fundrise
Copy link

When will this bug fix be released? Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport 1.8.latest bug Something isn't working empty Issues related to the --empty CLI flag user docs [docs.getdbt.com] Needs better documentation
Projects
None yet
10 participants