Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Snowflake Quote config not respected. #326

Open
1 of 3 tasks
pempey opened this issue Nov 25, 2024 · 1 comment
Open
1 of 3 tasks

Snowflake Quote config not respected. #326

pempey opened this issue Nov 25, 2024 · 1 comment
Labels
bug Something isn't working triage

Comments

@pempey
Copy link

pempey commented Nov 25, 2024

Describe the bug

When using a database with a name that requires quoting, such as starting with a number, the stage_external_sources will fail when it tries to check for existing schema in the snowflake__create_external_schema

show terse schemas like '{{ source_node.schema }}' in database {{ source_node.database }} limit 1;

Steps to reproduce

Create an external table on a Snowflake database where the database name begins with a number.

Expected results

I would expect the query to respect the quoting rules set by the project

# dbt_project.yml
quote:
  database: true
  schema: false
show terse schemas like 'test_schema' in database "1_TEST_DATABASE" limit 1;

Actual results

# dbt_project.yml
quote:
  database: true
  schema: false
show terse schemas like 'test_schema' in database 1_TEST_DATABASE limit 1;

Screenshots and log output

�[0m11:46:15.750359 [debug] [MainThread]: On macro_stage_external_sources: show terse schemas like 'gcp_billing' in database PEMPEY_PREP limit 1

System information

The contents of your packages.yml file:

packages:
  - git: [email protected]:gitlab-data/snowflake_spend.git
    revision: "updates_for_dbt_1_1"
    warn-unpinned: false
  - git: [email protected]:gitlab-data/data-tests.git
    revision: "{{env_var('DATA_TEST_BRANCH')}}"
    warn-unpinned: false
  - package: dbt-labs/audit_helper
    version: 0.9.0
  - package: dbt-labs/dbt_utils
    version: 1.1.1
  - package: dbt-labs/snowplow
    version: 0.15.1
  - package: dbt-labs/dbt_external_tables
    version: 0.8.7
  - package: brooklyn-data/dbt_artifacts
    version: 2.6.3

Which database are you using dbt with?

  • redshift
  • snowflake
  • other (specify: ____________)

The output of dbt --version:

Core:
  - installed: 1.8.4
  - latest:    1.8.9 - Update available!

  Your version of dbt-core is out of date!
  You can find instructions for upgrading here:
  https://docs.getdbt.com/docs/installation

Plugins:
  - snowflake: 1.8.4 - Up to date!

The operating system you're using:
macOS 15.1.1 (24B91)

The output of python --version:
Python 3.10.3

Additional context

@pempey pempey added bug Something isn't working triage labels Nov 25, 2024
@pempey
Copy link
Author

pempey commented Nov 25, 2024

FYI I was able to get the desired functionality by replaceing the snowflake__create_external_schema macro with this:

{% macro snowflake__create_external_schema(source_node) %}

    {% set ddl = adapter.create_schema(api.Relation.create(database=source_node.database, schema=source_node.schema)) %}
    {% if not ddl %}
      {% set ddl %}
            select 'Schema {{ source_node.schema }} exists' from dual;
        {% endset %}
    {% endif %}

    {% do return(ddl) %}

{% endmacro %}

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working triage
Projects
None yet
Development

No branches or pull requests

1 participant