Merge branch 'main' into generic-tests-config-tests

.bumpversion.cfg

@@ -1,5 +1,5 @@
 [bumpversion]
-current_version = 1.9.0a1
+current_version = 1.9.0b1
 parse = (?P<major>[\d]+) # major version number
 	\.(?P<minor>[\d]+) # minor version number
 	\.(?P<patch>[\d]+) # patch version number

.changes/1.9.0-b1.md

@@ -0,0 +1,61 @@
+## dbt-snowflake 1.9.0-b1 - October 01, 2024
+
+### Features
+
+- Support refresh_mode and initialize parameters for dynamic tables ([#1076](https://github.com/dbt-labs/dbt-snowflake/issues/1076))
+- Add tests for cross-database `cast` macro ([#1009](https://github.com/dbt-labs/dbt-snowflake/issues/1009))
+- Cross-database `date` macro ([#1013](https://github.com/dbt-labs/dbt-snowflake/issues/1013))
+- Replace underscores with hyphens in account IDs to prevent SSL issues ([#1068](https://github.com/dbt-labs/dbt-snowflake/issues/1068))
+- Support JWT Authentication ([#1079](https://github.com/dbt-labs/dbt-snowflake/issues/1079), [#726](https://github.com/dbt-labs/dbt-snowflake/issues/726))
+- Improve run times for large projects by reusing connections by default ([#1082](https://github.com/dbt-labs/dbt-snowflake/issues/1082))
+- Improve run times when using key pair auth by caching the private key ([#1082](https://github.com/dbt-labs/dbt-snowflake/issues/1082))
+- Add support for Iceberg table materializations. ([#321](https://github.com/dbt-labs/dbt-snowflake/issues/321))
+- Microbatch incremental strategy ([#1182](https://github.com/dbt-labs/dbt-snowflake/issues/1182))
+- Add support for Iceberg table format in Dynamic Tables ([#1183](https://github.com/dbt-labs/dbt-snowflake/issues/1183))
+- Add Iceberg format Incremental Models ([#321](https://github.com/dbt-labs/dbt-snowflake/issues/321))
+- Add support for all on_schema_change incremental model strategies. ([#321](https://github.com/dbt-labs/dbt-snowflake/issues/321))
+
+### Fixes
+
+- Get catalog metadata for a single relation in the most optimized way using the get_catalog_for_single_relation macro and capability ([#1048](https://github.com/dbt-labs/dbt-snowflake/issues/1048))
+- Update relation caching to correctly identify dynamic tables, accounting for Snowflake's `2024_03` bundle ([#1016](https://github.com/dbt-labs/dbt-snowflake/issues/1016))
+- Rename targets for tables and views use fully qualified names ([#1031](https://github.com/dbt-labs/dbt-snowflake/issues/1031))
+- Surface SSO token expiration in logs ([#851](https://github.com/dbt-labs/dbt-snowflake/issues/851))
+- return to previous naming convention to return to quoting policy ([#1074](https://github.com/dbt-labs/dbt-snowflake/issues/1074))
+- Fix scenario where using the `--empty` flag causes metadata queries to contain limit clauses ([#1033](https://github.com/dbt-labs/dbt-snowflake/issues/1033))
+- Use show ... starts with instead of show ... like in _show_object_metadata ([#1102](https://github.com/dbt-labs/dbt-snowflake/issues/1102))
+- Fix issue where dbt-snowflake attempts to drop database roles during grants sync ([#1151](https://github.com/dbt-labs/dbt-snowflake/issues/1151))
+
+### Under the Hood
+
+- Lazy load agate ([#953](https://github.com/dbt-labs/dbt-snowflake/issues/953))
+- Speedup catalog string comparison by using ilike before equals ([#1035](https://github.com/dbt-labs/dbt-snowflake/issues/1035))
+- Improve memory efficiency of the process_results() override. ([#1053](https://github.com/dbt-labs/dbt-snowflake/issues/1053))
+- Automate all manual integration tests for Dynamic Tables ([#1084](https://github.com/dbt-labs/dbt-snowflake/issues/1084))
+- Add support for experimental record/replay testing. ([#1106](https://github.com/dbt-labs/dbt-snowflake/issues/1106))
+- Remove `freezegun` as a testing dependency; this package is no longer used ([#1136](https://github.com/dbt-labs/dbt-snowflake/issues/1136))
+- Add support for Python 3.12 ([#903](https://github.com/dbt-labs/dbt-snowflake/issues/903))
+- Isolating distribution testing ([#1130](https://github.com/dbt-labs/dbt-snowflake/issues/1130))
+- Change behavior flag semantics to log iceberg flag warnings.. ([#321](https://github.com/dbt-labs/dbt-snowflake/issues/321))
+
+### Dependencies
+
+- Update freezegun requirement from ~=1.3 to ~=1.4 ([#869](https://github.com/dbt-labs/dbt-snowflake/pull/869))
+- Bump actions/upload-artifact from 3 to 4 ([#971](https://github.com/dbt-labs/dbt-snowflake/pull/971))
+- Bump dbt-labs/actions from 1.1.0 to 1.1.1 ([#1006](https://github.com/dbt-labs/dbt-snowflake/pull/1006))
+- Bump actions/download-artifact from 3 to 4 ([#1007](https://github.com/dbt-labs/dbt-snowflake/pull/1007))
+- Bump aurelien-baudet/workflow-dispatch from 2 to 4 ([#1093](https://github.com/dbt-labs/dbt-snowflake/pull/1093))
+- Update twine requirement from ~=4.0 to ~=5.1 ([#1120](https://github.com/dbt-labs/dbt-snowflake/pull/1120))
+- Bump pre-commit from 3.7.0 to 3.7.1 ([#1119](https://github.com/dbt-labs/dbt-snowflake/pull/1119))
+- Update wheel requirement from ~=0.42 to ~=0.43 ([#1121](https://github.com/dbt-labs/dbt-snowflake/pull/1121))
+- Update pytest-xdist requirement from ~=3.5 to ~=3.6 ([#1122](https://github.com/dbt-labs/dbt-snowflake/pull/1122))
+- Update tox requirement from ~=4.11 to ~=4.16 ([#1135](https://github.com/dbt-labs/dbt-snowflake/pull/1135))
+
+### Contributors
+- [@HenkvanDyk,mikealfare](https://github.com/HenkvanDyk,mikealfare) ([#1076](https://github.com/dbt-labs/dbt-snowflake/issues/1076))
+- [@McKnight-42](https://github.com/McKnight-42) ([#851](https://github.com/dbt-labs/dbt-snowflake/issues/851), [#1074](https://github.com/dbt-labs/dbt-snowflake/issues/1074))
+- [@amardatar](https://github.com/amardatar) ([#1082](https://github.com/dbt-labs/dbt-snowflake/issues/1082))
+- [@dwreeves](https://github.com/dwreeves) ([#953](https://github.com/dbt-labs/dbt-snowflake/issues/953))
+- [@leahwicz](https://github.com/leahwicz) ([#1130](https://github.com/dbt-labs/dbt-snowflake/issues/1130))
+- [@llam15](https://github.com/llam15) ([#1079](https://github.com/dbt-labs/dbt-snowflake/issues/1079), [#726](https://github.com/dbt-labs/dbt-snowflake/issues/726))
+- [@mikealfare,](https://github.com/mikealfare,) ([#851](https://github.com/dbt-labs/dbt-snowflake/issues/851))

.changes/1.9.0/Features-20240917-100505.yaml

@@ -0,0 +1,6 @@
+kind: Features
+body: Add support for Iceberg table format in Dynamic Tables
+time: 2024-09-17T10:05:05.609859-04:00
+custom:
+  Author: mikealfare
+  Issue: "1183"

.changes/1.9.0/Features-20240923-203204.yaml

@@ -0,0 +1,6 @@
+kind: Features
+body: Add Iceberg format Incremental Models
+time: 2024-09-23T20:32:04.783741-07:00
+custom:
+    Author: versusfacit
+    Issue: "321"

.changes/1.9.0/Features-20240930-112041.yaml

@@ -0,0 +1,6 @@
+kind: Features
+body: Add support for all on_schema_change incremental model strategies.
+time: 2024-09-30T11:20:41.99589-07:00
+custom:
+    Author: versusfacit
+    Issue: "321"

.changes/1.9.0/Fixes-20240920-193613.yaml

@@ -0,0 +1,6 @@
+kind: Fixes
+body: Fix issue where dbt-snowflake attempts to drop database roles during grants sync
+time: 2024-09-20T19:36:13.671173-04:00
+custom:
+  Author: mikealfare
+  Issue: "1151"

.changes/unreleased/Breaking Changes-20241016-183143.yaml

@@ -0,0 +1,6 @@
+kind: Breaking Changes
+body: Drop support for Python 3.8
+time: 2024-10-16T18:31:43.4167-04:00
+custom:
+  Author: mikealfare
+  Issue: "1211"

.changes/unreleased/Fixes-20241008-122635.yaml

@@ -0,0 +1,6 @@
+kind: Fixes
+body: Dynamic Iceberg table base_location_subpath generation fix.
+time: 2024-10-08T12:26:35.521308-07:00
+custom:
+    Author: versusfacit
+    Issue: "1200"

.github/scripts/integration-test-matrix.js

@@ -1,6 +1,6 @@
 module.exports = ({ context }) => {
-  const defaultPythonVersion = "3.8";
-  const supportedPythonVersions = ["3.8", "3.9", "3.10", "3.11", "3.12"];
+  const defaultPythonVersion = "3.9";
+  const supportedPythonVersions = ["3.9", "3.10", "3.11", "3.12"];
   const supportedAdapters = ["snowflake"];
 
   // if PR, generate matrix based on files changed and PR labels

.github/workflows/main.yml

@@ -50,7 +50,7 @@ jobs:
       - name: Set up Python
         uses: actions/setup-python@v5
         with:
-          python-version: '3.8'
+          python-version: '3.9'
 
       - name: Install python dependencies
         run: |
@@ -71,7 +71,7 @@ jobs:
     strategy:
       fail-fast: false
       matrix:
-        python-version: ['3.8', '3.9', '3.10', '3.11', '3.12']
+        python-version: ['3.9', '3.10', '3.11', '3.12']
 
     env:
       TOXENV: "unit"
@@ -127,7 +127,7 @@ jobs:
       - name: Set up Python
         uses: actions/setup-python@v5
         with:
-          python-version: '3.8'
+          python-version: '3.9'
 
       - name: Install python dependencies
         run: |
@@ -175,7 +175,7 @@ jobs:
       fail-fast: false
       matrix:
         os: [ubuntu-latest, macos-12, windows-latest]
-        python-version: ['3.8', '3.9', '3.10', '3.11', '3.12']
+        python-version: ['3.9', '3.10', '3.11', '3.12']
         dist-type: ['whl', 'gz']
 
     steps:

.pre-commit-config.yaml

@@ -24,7 +24,6 @@ repos:
     -   id: black
         args:
         -   --line-length=99
-        -   --target-version=py38
         -   --target-version=py39
         -   --target-version=py310
         -   --target-version=py311

CHANGELOG.md

@@ -5,10 +5,71 @@
 - "Breaking changes" listed under a version may require action from end users or external maintainers when upgrading to that version.
 - Do not edit this file directly. This file is auto-generated using [changie](https://github.com/miniscruff/changie). For details on how to document a change, see [the contributing guide](https://github.com/dbt-labs/dbt-snowflake/blob/main/CONTRIBUTING.md#adding-changelog-entry)
 
+## dbt-snowflake 1.9.0-b1 - October 01, 2024
+
+### Features
+
+- Support refresh_mode and initialize parameters for dynamic tables ([#1076](https://github.com/dbt-labs/dbt-snowflake/issues/1076))
+- Add tests for cross-database `cast` macro ([#1009](https://github.com/dbt-labs/dbt-snowflake/issues/1009))
+- Cross-database `date` macro ([#1013](https://github.com/dbt-labs/dbt-snowflake/issues/1013))
+- Replace underscores with hyphens in account IDs to prevent SSL issues ([#1068](https://github.com/dbt-labs/dbt-snowflake/issues/1068))
+- Support JWT Authentication ([#1079](https://github.com/dbt-labs/dbt-snowflake/issues/1079), [#726](https://github.com/dbt-labs/dbt-snowflake/issues/726))
+- Improve run times for large projects by reusing connections by default ([#1082](https://github.com/dbt-labs/dbt-snowflake/issues/1082))
+- Improve run times when using key pair auth by caching the private key ([#1082](https://github.com/dbt-labs/dbt-snowflake/issues/1082))
+- Add support for Iceberg table materializations. ([#321](https://github.com/dbt-labs/dbt-snowflake/issues/321))
+- Microbatch incremental strategy ([#1182](https://github.com/dbt-labs/dbt-snowflake/issues/1182))
+- Add support for Iceberg table format in Dynamic Tables ([#1183](https://github.com/dbt-labs/dbt-snowflake/issues/1183))
+- Add Iceberg format Incremental Models ([#321](https://github.com/dbt-labs/dbt-snowflake/issues/321))
+- Add support for all on_schema_change incremental model strategies. ([#321](https://github.com/dbt-labs/dbt-snowflake/issues/321))
+
+### Fixes
+
+- Get catalog metadata for a single relation in the most optimized way using the get_catalog_for_single_relation macro and capability ([#1048](https://github.com/dbt-labs/dbt-snowflake/issues/1048))
+- Update relation caching to correctly identify dynamic tables, accounting for Snowflake's `2024_03` bundle ([#1016](https://github.com/dbt-labs/dbt-snowflake/issues/1016))
+- Rename targets for tables and views use fully qualified names ([#1031](https://github.com/dbt-labs/dbt-snowflake/issues/1031))
+- Surface SSO token expiration in logs ([#851](https://github.com/dbt-labs/dbt-snowflake/issues/851))
+- return to previous naming convention to return to quoting policy ([#1074](https://github.com/dbt-labs/dbt-snowflake/issues/1074))
+- Fix scenario where using the `--empty` flag causes metadata queries to contain limit clauses ([#1033](https://github.com/dbt-labs/dbt-snowflake/issues/1033))
+- Use show ... starts with instead of show ... like in _show_object_metadata ([#1102](https://github.com/dbt-labs/dbt-snowflake/issues/1102))
+- Fix issue where dbt-snowflake attempts to drop database roles during grants sync ([#1151](https://github.com/dbt-labs/dbt-snowflake/issues/1151))
+
+### Under the Hood
+
+- Lazy load agate ([#953](https://github.com/dbt-labs/dbt-snowflake/issues/953))
+- Speedup catalog string comparison by using ilike before equals ([#1035](https://github.com/dbt-labs/dbt-snowflake/issues/1035))
+- Improve memory efficiency of the process_results() override. ([#1053](https://github.com/dbt-labs/dbt-snowflake/issues/1053))
+- Automate all manual integration tests for Dynamic Tables ([#1084](https://github.com/dbt-labs/dbt-snowflake/issues/1084))
+- Add support for experimental record/replay testing. ([#1106](https://github.com/dbt-labs/dbt-snowflake/issues/1106))
+- Remove `freezegun` as a testing dependency; this package is no longer used ([#1136](https://github.com/dbt-labs/dbt-snowflake/issues/1136))
+- Add support for Python 3.12 ([#903](https://github.com/dbt-labs/dbt-snowflake/issues/903))
+- Isolating distribution testing ([#1130](https://github.com/dbt-labs/dbt-snowflake/issues/1130))
+- Change behavior flag semantics to log iceberg flag warnings.. ([#321](https://github.com/dbt-labs/dbt-snowflake/issues/321))
+
+### Dependencies
+
+- Update freezegun requirement from ~=1.3 to ~=1.4 ([#869](https://github.com/dbt-labs/dbt-snowflake/pull/869))
+- Bump actions/upload-artifact from 3 to 4 ([#971](https://github.com/dbt-labs/dbt-snowflake/pull/971))
+- Bump dbt-labs/actions from 1.1.0 to 1.1.1 ([#1006](https://github.com/dbt-labs/dbt-snowflake/pull/1006))
+- Bump actions/download-artifact from 3 to 4 ([#1007](https://github.com/dbt-labs/dbt-snowflake/pull/1007))
+- Bump aurelien-baudet/workflow-dispatch from 2 to 4 ([#1093](https://github.com/dbt-labs/dbt-snowflake/pull/1093))
+- Update twine requirement from ~=4.0 to ~=5.1 ([#1120](https://github.com/dbt-labs/dbt-snowflake/pull/1120))
+- Bump pre-commit from 3.7.0 to 3.7.1 ([#1119](https://github.com/dbt-labs/dbt-snowflake/pull/1119))
+- Update wheel requirement from ~=0.42 to ~=0.43 ([#1121](https://github.com/dbt-labs/dbt-snowflake/pull/1121))
+- Update pytest-xdist requirement from ~=3.5 to ~=3.6 ([#1122](https://github.com/dbt-labs/dbt-snowflake/pull/1122))
+- Update tox requirement from ~=4.11 to ~=4.16 ([#1135](https://github.com/dbt-labs/dbt-snowflake/pull/1135))
+
+### Contributors
+- [@HenkvanDyk,mikealfare](https://github.com/HenkvanDyk,mikealfare) ([#1076](https://github.com/dbt-labs/dbt-snowflake/issues/1076))
+- [@McKnight-42](https://github.com/McKnight-42) ([#851](https://github.com/dbt-labs/dbt-snowflake/issues/851), [#1074](https://github.com/dbt-labs/dbt-snowflake/issues/1074))
+- [@amardatar](https://github.com/amardatar) ([#1082](https://github.com/dbt-labs/dbt-snowflake/issues/1082))
+- [@dwreeves](https://github.com/dwreeves) ([#953](https://github.com/dbt-labs/dbt-snowflake/issues/953))
+- [@leahwicz](https://github.com/leahwicz) ([#1130](https://github.com/dbt-labs/dbt-snowflake/issues/1130))
+- [@llam15](https://github.com/llam15) ([#1079](https://github.com/dbt-labs/dbt-snowflake/issues/1079), [#726](https://github.com/dbt-labs/dbt-snowflake/issues/726))
+- [@mikealfare,](https://github.com/mikealfare,) ([#851](https://github.com/dbt-labs/dbt-snowflake/issues/851))
+
+
 ## Previous Releases
 For information on prior major and minor releases, see their changelogs:
-- [1.8](https://github.com/dbt-labs/dbt-snowflake/blob/1.8.latest/CHANGELOG.md)
-- [1.7](https://github.com/dbt-labs/dbt-snowflake/blob/1.7.latest/CHANGELOG.md)
 - [1.6](https://github.com/dbt-labs/dbt-snowflake/blob/1.6.latest/CHANGELOG.md)
 - [1.5](https://github.com/dbt-labs/dbt-snowflake/blob/1.5.latest/CHANGELOG.md)
 - [1.4](https://github.com/dbt-labs/dbt-snowflake/blob/1.4.latest/CHANGELOG.md)

CONTRIBUTING.md

@@ -79,7 +79,7 @@ WARNING: The parameters in your `test.env` file must link to a valid Snowflake a
 There are a few methods for running tests locally.
 
 #### `tox`
-`tox` automatically runs unit tests against several Python versions using its own virtualenvs. Run `tox -p` to run unit tests for Python 3.8, Python 3.9, Python 3.10, and `flake8` in parallel. Run `tox -e py38` to invoke tests on Python version 3.8 only (use py38, py39, or py310). Tox recipes are found in `tox.ini`.
+`tox` automatically runs unit tests against several Python versions using its own virtualenvs. Run `tox -p` to run unit tests for Python 3.9 and Python 3.10, and `flake8` in parallel. Run `tox -e py39` to invoke tests on Python version 3.9 only (use py39 or py310). Tox recipes are found in `tox.ini`.
 
 #### `pytest`
 You may run a specific test or group of tests using `pytest` directly. Activate a Python virtualenv active with dev dependencies installed. Then, run tests like so:

Makefile

@@ -12,20 +12,20 @@ dev-uninstall: ## Uninstalls all packages while maintaining the virtual environm
 	pip uninstall -y dbt-snowflake
 
 .PHONY: unit
-unit: ## Runs unit tests with py38.
+unit: ## Runs unit tests with py39.
 	@\
-	tox -e py38
+	tox -e py39
 
 .PHONY: test
-test: ## Runs unit tests with py38 and code checks against staged changes.
+test: ## Runs unit tests with py39 and code checks against staged changes.
 	@\
-	tox -p -e py38; \
+	tox -p -e py39; \
 	pre-commit run --all-files
 
 .PHONY: integration
 integration: ## Runs snowflake integration tests with py38.
 	@\
-	tox -e py38-snowflake --
+	tox -e py39-snowflake --
 
 .PHONY: clean
 	@echo "cleaning repo"

dbt/adapters/snowflake/__version__.py

@@ -1 +1 @@
-version = "1.9.0a1"
+version = "1.9.0b1"

dbt/adapters/snowflake/impl.py

@@ -320,7 +320,7 @@ def standardize_grants_dict(self, grants_table: "agate.Table") -> dict:
             grantee = row["grantee_name"]
             granted_to = row["granted_to"]
             privilege = row["privilege"]
-            if privilege != "OWNERSHIP" and granted_to != "SHARE":
+            if privilege != "OWNERSHIP" and granted_to not in ["SHARE", "DATABASE_ROLE"]:
                 if privilege in grants_dict.keys():
                     grants_dict[privilege].append(grantee)
                 else:

dbt/adapters/snowflake/relation.py

@@ -17,6 +17,7 @@
 from dbt_common.events.functions import fire_event, warn_or_error
 
 from dbt.adapters.snowflake.relation_configs import (
+    SnowflakeCatalogConfigChange,
     SnowflakeDynamicTableConfig,
     SnowflakeDynamicTableConfigChangeset,
     SnowflakeDynamicTableRefreshModeConfigChange,
@@ -114,6 +115,12 @@ def dynamic_table_config_changeset(
                 context=new_dynamic_table.refresh_mode,
             )
 
+        if new_dynamic_table.catalog != existing_dynamic_table.catalog:
+            config_change_collection.catalog = SnowflakeCatalogConfigChange(
+                action=RelationConfigChangeAction.create,
+                context=new_dynamic_table.catalog,
+            )
+
         if config_change_collection.has_changes:
             return config_change_collection
         return None
@@ -132,6 +139,14 @@ def as_case_sensitive(self) -> "SnowflakeRelation":
 
         return self.replace_path(**path_part_map)
 
+    @property
+    def can_be_renamed(self) -> bool:
+        """
+        Standard tables and dynamic tables can be renamed, but Snowflake does not support renaming iceberg relations.
+        The iceberg standard does support renaming, so this may change in the future.
+        """
+        return self.type in self.renameable_relations and not self.is_iceberg_format
+
     def get_ddl_prefix_for_create(self, config: RelationConfig, temporary: bool) -> str:
         """
         This macro renders the appropriate DDL prefix during the create_table_as

dbt/adapters/snowflake/relation_configs/__init__.py

@@ -1,13 +1,17 @@
+from dbt.adapters.snowflake.relation_configs.catalog import (
+    SnowflakeCatalogConfig,
+    SnowflakeCatalogConfigChange,
+)
 from dbt.adapters.snowflake.relation_configs.dynamic_table import (
     SnowflakeDynamicTableConfig,
     SnowflakeDynamicTableConfigChangeset,
     SnowflakeDynamicTableRefreshModeConfigChange,
     SnowflakeDynamicTableWarehouseConfigChange,
     SnowflakeDynamicTableTargetLagConfigChange,
 )
+from dbt.adapters.snowflake.relation_configs.formats import TableFormat
 from dbt.adapters.snowflake.relation_configs.policies import (
     SnowflakeIncludePolicy,
     SnowflakeQuotePolicy,
     SnowflakeRelationType,
 )
-from dbt.adapters.snowflake.relation_configs.formats import TableFormat