Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

first pass: unit test typing #839

Closed
wants to merge 4 commits into from
+34 −3
Closed

first pass: unit test typing #839

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
a686ee8
Init unit_testing tests
jtcohen6 Nov 21, 2023
7173c51
Repoint dbt-core dev branch
jtcohen6 Nov 21, 2023
a87c136
Update TYPE_LABELS + safe_cast
jtcohen6 Nov 21, 2023
13eb2d2
Fix model contract tests
jtcohen6 Nov 22, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions dbt/adapters/snowflake/column.py
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,11 +1,16 @@
from dataclasses import dataclass
from typing import Dict, ClassVar

from dbt.adapters.base.column import Column
from dbt.exceptions import DbtRuntimeError


@dataclass
class SnowflakeColumn(Column):
TYPE_LABELS: ClassVar[Dict[str, str]] = {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The parent/default Column.TYPE_LABELS has STRING -> TEXT. I haven't included it here because these are true aliases on Snowflake.

"FIXED": "NUMERIC",
Copy link
Contributor Author

@jtcohen6 jtcohen6 Nov 22, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note this means model contract errors will now say NUMERIC instead of FIXED — and they will say this for all fixed-precision numeric types, including integers, because INT on Snowflake is just an alias for NUMERIC(38,0).

Compilation Error in model my_model (models/my_model.sql)
  This model has an enforced contract that failed.
  Please ensure the name, data_type, and number of columns in your contract match the columns in your model's definition.

  | column_name | definition_type | contract_type | mismatch_reason    |
  | ----------- | --------------- | ------------- | ------------------ |
  | ID          | NUMERIC         | TEXT          | data type mismatch |

This is more helpful IMO, because numeric is a real type that Snowflake will actually allow you to write in SQL!

Floating-point types will still appear as REAL, and dbt will detect mismatch:

Compilation Error in model my_model (models/my_model.sql)
  This model has an enforced contract that failed.
  Please ensure the name, data_type, and number of columns in your contract match the columns in your model's definition.

  | column_name | definition_type | contract_type | mismatch_reason    |
  | ----------- | --------------- | ------------- | ------------------ |
  | ID          | NUMERIC         | REAL          | data type mismatch |

}

def is_integer(self) -> bool:
# everything that smells like an int is actually a NUMBER(38, 0)
return False
Expand Down
7 changes: 7 additions & 0 deletions dbt/include/snowflake/macros/utils/safe_cast.sql
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,3 +1,10 @@
{% macro snowflake__safe_cast(field, type) %}
{#-- Dumb for now --#}
{% if type|lower in ['array', 'variant'] and field is string %}
cast(try_parse_json({{field}}) as {{type}})
{% elif field is string %}
try_cast({{field}} as {{type}})
{% else %}
cast({{field}} as {{type}})
{% endif %}
{% endmacro %}
4 changes: 2 additions & 2 deletions dev-requirements.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# install latest changes in dbt-core
# TODO: how to automate switching from develop to version branches?
git+https://github.com/dbt-labs/dbt-core.git#egg=dbt-core&subdirectory=core
git+https://github.com/dbt-labs/dbt-core.git#egg=dbt-tests-adapter&subdirectory=tests/adapter
git+https://github.com/dbt-labs/dbt-core.git@jerco/unit-test-case-insensitivity#egg=dbt-core&subdirectory=core
git+https://github.com/dbt-labs/dbt-core.git@jerco/unit-test-case-insensitivity#egg=dbt-tests-adapter&subdirectory=tests/adapter

# if version 1.x or greater -> pin to major version
# if version 0.x -> pin to minor
Expand Down
2 changes: 1 addition & 1 deletion tests/functional/adapter/test_constraints.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -66,7 +66,7 @@
class SnowflakeColumnEqualSetup:
@pytest.fixture
def int_type(self):
return "FIXED"
return "NUMERIC"

@pytest.fixture
def schema_int_type(self):
Expand Down
19 changes: 19 additions & 0 deletions tests/functional/adapter/unit_testing/test_unit_testing_types.py
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
import pytest
from dbt.tests.adapter.unit_testing.test_unit_testing_types import BaseUnitTestingTypes


class TestBigQueryUnitTestingTypes(BaseUnitTestingTypes):
@pytest.fixture
def data_types(self):
# sql_value, yaml_value
return [
["1", "1"],
["'1'", "1"],
["cast('true' as boolean)", "true"],
["cast('2019-01-01' as date)", "2019-01-01"],
["cast('2013-11-03 00:00:00-07' as TIMESTAMP)", "2013-11-03 00:00:00-07"],
["['a','b','c']", "['a','b','c']"],
["[1,2,3]", "[1,2,3]"],
["cast(1 as NUMERIC)", "1"],
["""'{"name": "Cooper", "forname": "Alice"}'""", """'{"name": "Cooper", "forname": "Alice"}'"""],
]
Loading