Merge branch 'main' into state-modified-vars

.bumpversion.cfg

@@ -1,5 +1,5 @@
 [bumpversion]
-current_version = 1.9.0a1
+current_version = 1.9.0b4
 parse = (?P<major>[\d]+) # major version number
 	\.(?P<minor>[\d]+) # minor version number
 	\.(?P<patch>[\d]+) # patch version number

.changes/1.9.0-b1.md

@@ -0,0 +1,136 @@
+## dbt-core 1.9.0-b1 - October 01, 2024
+
+### Breaking Changes
+
+- Fix changing the current working directory when using dpt deps, clean and init. ([#8997](https://github.com/dbt-labs/dbt-core/issues/8997))
+
+### Features
+
+- serialize inferred primary key ([#9824](https://github.com/dbt-labs/dbt-core/issues/9824))
+- Add unit_test: selection method ([#10053](https://github.com/dbt-labs/dbt-core/issues/10053))
+- Maximally parallelize dbt clone in clone command" ([#7914](https://github.com/dbt-labs/dbt-core/issues/7914))
+- Add --host flag to dbt docs serve, defaulting to '127.0.0.1' ([#10229](https://github.com/dbt-labs/dbt-core/issues/10229))
+- Update data_test to accept arbitrary config options ([#10197](https://github.com/dbt-labs/dbt-core/issues/10197))
+- add pre_model and post_model hook calls to data and unit tests to be able to provide extra config options ([#10198](https://github.com/dbt-labs/dbt-core/issues/10198))
+- add --empty value to jinja context as flags.EMPTY ([#10317](https://github.com/dbt-labs/dbt-core/issues/10317))
+- Warning message for snapshot timestamp data types ([#10234](https://github.com/dbt-labs/dbt-core/issues/10234))
+- Support cumulative_type_params & sub-daily granularities in semantic manifest. ([#10360](https://github.com/dbt-labs/dbt-core/issues/10360))
+- Add time_granularity to metric spec. ([#10376](https://github.com/dbt-labs/dbt-core/issues/10376))
+- Support standard schema/database fields for snapshots ([#10301](https://github.com/dbt-labs/dbt-core/issues/10301))
+- Support ref and source in foreign key constraint expressions, bump dbt-common minimum to 1.6 ([#8062](https://github.com/dbt-labs/dbt-core/issues/8062))
+- Support new semantic layer time spine configs to enable sub-daily granularity. ([#10475](https://github.com/dbt-labs/dbt-core/issues/10475))
+- Include models that depend on changed vars in state:modified, add state:modified.vars selection method ([#4304](https://github.com/dbt-labs/dbt-core/issues/4304))
+- Add support for behavior flags ([#10618](https://github.com/dbt-labs/dbt-core/issues/10618))
+- Enable `--resource-type` and `--exclude-resource-type` CLI flags and environment variables for `dbt test` ([#10656](https://github.com/dbt-labs/dbt-core/issues/10656))
+- Allow configuring snapshot column names ([#10185](https://github.com/dbt-labs/dbt-core/issues/10185))
+- Add custom_granularities to YAML spec for time spines. ([#9265](https://github.com/dbt-labs/dbt-core/issues/9265))
+- Add basic functionality for creating microbatch incremental models ([#9490](https://github.com/dbt-labs/dbt-core/issues/9490), [#10635](https://github.com/dbt-labs/dbt-core/issues/10635), [#10637](https://github.com/dbt-labs/dbt-core/issues/10637), [#10638](https://github.com/dbt-labs/dbt-core/issues/10638), [#10636](https://github.com/dbt-labs/dbt-core/issues/10636), [#10662](https://github.com/dbt-labs/dbt-core/issues/10662), [#10639](https://github.com/dbt-labs/dbt-core/issues/10639))
+- Execute microbatch models in batches ([#10700](https://github.com/dbt-labs/dbt-core/issues/10700))
+- Create 'skip_nodes_if_on_run_start_fails' behavior change flag ([#7387](https://github.com/dbt-labs/dbt-core/issues/7387))
+- Allow snapshots to be defined in YAML. ([#10246](https://github.com/dbt-labs/dbt-core/issues/10246))
+- Write microbatch compiled/run targets to separate files, one per batch ([#10714](https://github.com/dbt-labs/dbt-core/issues/10714))
+- Track incremental_strategy as part of model_run tracking event ([#10761](https://github.com/dbt-labs/dbt-core/issues/10761))
+- Support required 'begin' config for microbatch models ([#10701](https://github.com/dbt-labs/dbt-core/issues/10701))
+- Parse-time validation of microbatch configs: require event_time, batch_size, lookback and validate input event_time ([#10709](https://github.com/dbt-labs/dbt-core/issues/10709))
+- Added the --inline-direct parameter to 'dbt show' ([#10770](https://github.com/dbt-labs/dbt-core/issues/10770))
+- Enable `retry` support for microbatch models ([#10715](https://github.com/dbt-labs/dbt-core/issues/10715), [#10729](https://github.com/dbt-labs/dbt-core/issues/10729))
+- Use unrendered database and schema source properties during state:modified, behind state_modified_compare_more_unrendered_values behavoiur flag ([#9573](https://github.com/dbt-labs/dbt-core/issues/9573))
+- Ensure microbatch models respect `full_refresh` model config ([#10785](https://github.com/dbt-labs/dbt-core/issues/10785))
+- Adds validations for custom_granularities to ensure unique naming. ([#9265](https://github.com/dbt-labs/dbt-core/issues/9265))
+
+### Fixes
+
+- Remove unused check_new method ([#7586](https://github.com/dbt-labs/dbt-core/issues/7586))
+- Test case for `merge_exclude_columns` ([#8267](https://github.com/dbt-labs/dbt-core/issues/8267))
+- Convert "Skipping model due to fail_fast" message to DEBUG level ([#8774](https://github.com/dbt-labs/dbt-core/issues/8774))
+- Restore previous behavior for --favor-state: only favor defer_relation if not selected in current command" ([#10107](https://github.com/dbt-labs/dbt-core/issues/10107))
+- Unit test fixture (csv) returns null for empty value ([#9881](https://github.com/dbt-labs/dbt-core/issues/9881))
+- Fix json format log and --quiet for ls and jinja print by converting print call to fire events ([#8756](https://github.com/dbt-labs/dbt-core/issues/8756))
+- Add resource type to saved_query ([#10168](https://github.com/dbt-labs/dbt-core/issues/10168))
+- Fix: Order-insensitive unit test equality assertion for expected/actual with multiple nulls ([#10167](https://github.com/dbt-labs/dbt-core/issues/10167))
+- Renaming or removing a contracted model should raise a BreakingChange warning/error ([#10116](https://github.com/dbt-labs/dbt-core/issues/10116))
+- prefer disabled project nodes to external node ([#10224](https://github.com/dbt-labs/dbt-core/issues/10224))
+- Fix issues with selectors and inline nodes ([#8943](https://github.com/dbt-labs/dbt-core/issues/8943), [#9269](https://github.com/dbt-labs/dbt-core/issues/9269))
+- Fix snapshot config to work in yaml files ([#4000](https://github.com/dbt-labs/dbt-core/issues/4000))
+- Improve handling of error when loading schema file list ([#10284](https://github.com/dbt-labs/dbt-core/issues/10284))
+- Use model alias for the CTE identifier generated during ephemeral materialization ([#5273](https://github.com/dbt-labs/dbt-core/issues/5273))
+- Implement state:modified for saved queries ([#10294](https://github.com/dbt-labs/dbt-core/issues/10294))
+- Saved Query node fail during skip ([#10029](https://github.com/dbt-labs/dbt-core/issues/10029))
+- DOn't warn on `unit_test` config paths that are properly used ([#10311](https://github.com/dbt-labs/dbt-core/issues/10311))
+- Fix setting `silence` of `warn_error_options` via `dbt_project.yaml` flags ([#10160](https://github.com/dbt-labs/dbt-core/issues/10160))
+- Attempt to provide test fixture tables with all values to set types correctly for comparisong with source tables ([#10365](https://github.com/dbt-labs/dbt-core/issues/10365))
+- Limit data_tests deprecation to root_project ([#9835](https://github.com/dbt-labs/dbt-core/issues/9835))
+- CLI flags should take precedence over env var flags ([#10304](https://github.com/dbt-labs/dbt-core/issues/10304))
+- Fix typing for artifact schemas ([#10442](https://github.com/dbt-labs/dbt-core/issues/10442))
+- Fix over deletion of generated_metrics in partial parsing ([#10450](https://github.com/dbt-labs/dbt-core/issues/10450))
+- Fix error constructing warn_error_options ([#10452](https://github.com/dbt-labs/dbt-core/issues/10452))
+- Do not update varchar column definitions if a contract exists ([#10362](https://github.com/dbt-labs/dbt-core/issues/10362))
+- fix all_constraints access, disabled node parsing of non-uniquely named resources ([#10509](https://github.com/dbt-labs/dbt-core/issues/10509))
+- respect --quiet and --warn-error-options for flag deprecations ([#10105](https://github.com/dbt-labs/dbt-core/issues/10105))
+- Propagate measure label when using create_metrics ([#10536](https://github.com/dbt-labs/dbt-core/issues/10536))
+- Fix state:modified check for exports ([#10138](https://github.com/dbt-labs/dbt-core/issues/10138))
+- Filter out empty nodes after graph selection to support consistent selection of nodes that depend on upstream public models ([#8987](https://github.com/dbt-labs/dbt-core/issues/8987))
+- Late render pre- and post-hooks configs in properties / schema YAML files ([#10603](https://github.com/dbt-labs/dbt-core/issues/10603))
+- Allow the use of env_var function in certain macros in which it was previously unavailable. ([#10609](https://github.com/dbt-labs/dbt-core/issues/10609))
+- Remove deprecation for tests: to data_tests: change ([#10564](https://github.com/dbt-labs/dbt-core/issues/10564))
+- Fix `--resource-type test` for `dbt list` and `dbt build` ([#10730](https://github.com/dbt-labs/dbt-core/issues/10730))
+- Fix unit tests for incremental model with alias ([#10754](https://github.com/dbt-labs/dbt-core/issues/10754))
+- Allow singular tests to be documented in properties.yml ([#9005](https://github.com/dbt-labs/dbt-core/issues/9005))
+- Ignore --empty in unit test ref/source rendering ([#10516](https://github.com/dbt-labs/dbt-core/issues/10516))
+- Ignore rendered jinja in configs for state:modified, behind state_modified_compare_more_unrendered_values behaviour flag ([#9564](https://github.com/dbt-labs/dbt-core/issues/9564))
+- Improve performance of infer primary key ([#10781](https://github.com/dbt-labs/dbt-core/issues/10781))
+- Attempt to skip saved query processing when no semantic manifest changes ([#10563](https://github.com/dbt-labs/dbt-core/issues/10563))
+- Ensure dbt retry of microbatch models doesn't lose prior successful state ([#10800](https://github.com/dbt-labs/dbt-core/issues/10800))
+
+### Docs
+
+- Enable display of unit tests ([dbt-docs/#501](https://github.com/dbt-labs/dbt-docs/issues/501))
+- Unit tests not rendering ([dbt-docs/#506](https://github.com/dbt-labs/dbt-docs/issues/506))
+- Add support for Saved Query node ([dbt-docs/#486](https://github.com/dbt-labs/dbt-docs/issues/486))
+- Fix npm security vulnerabilities as of June 2024 ([dbt-docs/#513](https://github.com/dbt-labs/dbt-docs/issues/513))
+
+### Under the Hood
+
+- Clear error message for Private package in dbt-core ([#10083](https://github.com/dbt-labs/dbt-core/issues/10083))
+- Enable use of context in serialization ([#10093](https://github.com/dbt-labs/dbt-core/issues/10093))
+- Make RSS high water mark measurement more accurate on Linux ([#10177](https://github.com/dbt-labs/dbt-core/issues/10177))
+- Enable record filtering by type. ([#10240](https://github.com/dbt-labs/dbt-core/issues/10240))
+- Remove IntermediateSnapshotNode ([#10326](https://github.com/dbt-labs/dbt-core/issues/10326))
+- Additional logging for skipped ephemeral models ([#10389](https://github.com/dbt-labs/dbt-core/issues/10389))
+- bump black to 24.3.0 ([#10454](https://github.com/dbt-labs/dbt-core/issues/10454))
+- generate protos with protoc version 5.26.1 ([#10457](https://github.com/dbt-labs/dbt-core/issues/10457))
+- Move from minimal-snowplow-tracker fork back to snowplow-tracker ([#8409](https://github.com/dbt-labs/dbt-core/issues/8409))
+- Add group info to RunResultError, RunResultFailure, RunResultWarning log lines ([#](https://github.com/dbt-labs/dbt-core/issues/))
+- Improve speed of tree traversal when finding children, increasing build speed for some selectors ([#10434](https://github.com/dbt-labs/dbt-core/issues/10434))
+- Add test for sources tables with quotes ([#10582](https://github.com/dbt-labs/dbt-core/issues/10582))
+- Additional type hints for `core/dbt/version.py` ([#10612](https://github.com/dbt-labs/dbt-core/issues/10612))
+- Fix typing issues in core/dbt/contracts/sql.py ([#10614](https://github.com/dbt-labs/dbt-core/issues/10614))
+- Fix type errors in `dbt/core/task/clean.py` ([#10616](https://github.com/dbt-labs/dbt-core/issues/10616))
+- Add Snowplow tracking for behavior flag deprecations ([#10552](https://github.com/dbt-labs/dbt-core/issues/10552))
+- Add test utility patch_microbatch_end_time for adapters testing ([#10713](https://github.com/dbt-labs/dbt-core/issues/10713))
+- Replace `TestSelector` with `ResourceTypeSelector` ([#10718](https://github.com/dbt-labs/dbt-core/issues/10718))
+- Standardize returning `ResourceTypeSelector` instances in `dbt list` and `dbt build` ([#10739](https://github.com/dbt-labs/dbt-core/issues/10739))
+- Add group metadata info to LogModelResult and LogTestResult ([#10775](https://github.com/dbt-labs/dbt-core/issues/10775))
+
+### Dependencies
+
+- Remove logbook dependency ([#8027](https://github.com/dbt-labs/dbt-core/issues/8027))
+- Increase supported version range for dbt-semantic-interfaces. Needed to support custom calendar features. ([#9265](https://github.com/dbt-labs/dbt-core/issues/9265))
+
+### Security
+
+- Explicitly bind to localhost in docs serve ([#10209](https://github.com/dbt-labs/dbt-core/issues/10209))
+
+### Contributors
+- [@McKnight-42](https://github.com/McKnight-42) ([#10197](https://github.com/dbt-labs/dbt-core/issues/10197), [#10198](https://github.com/dbt-labs/dbt-core/issues/10198))
+- [@TowardOliver](https://github.com/TowardOliver) ([#10656](https://github.com/dbt-labs/dbt-core/issues/10656))
+- [@aliceliu](https://github.com/aliceliu) ([#10536](https://github.com/dbt-labs/dbt-core/issues/10536), [#10138](https://github.com/dbt-labs/dbt-core/issues/10138))
+- [@courtneyholcomb](https://github.com/courtneyholcomb) ([#10360](https://github.com/dbt-labs/dbt-core/issues/10360), [#10376](https://github.com/dbt-labs/dbt-core/issues/10376), [#10475](https://github.com/dbt-labs/dbt-core/issues/10475), [#9265](https://github.com/dbt-labs/dbt-core/issues/9265), [#9265](https://github.com/dbt-labs/dbt-core/issues/9265), [#9265](https://github.com/dbt-labs/dbt-core/issues/9265))
+- [@dave-connors-3](https://github.com/dave-connors-3) ([#9824](https://github.com/dbt-labs/dbt-core/issues/9824))
+- [@jeancochrane](https://github.com/jeancochrane) ([#5273](https://github.com/dbt-labs/dbt-core/issues/5273))
+- [@katsugeneration](https://github.com/katsugeneration) ([#10754](https://github.com/dbt-labs/dbt-core/issues/10754))
+- [@kevinneville](https://github.com/kevinneville) ([#7586](https://github.com/dbt-labs/dbt-core/issues/7586))
+- [@nakamichiworks](https://github.com/nakamichiworks) ([#10442](https://github.com/dbt-labs/dbt-core/issues/10442))
+- [@rariyama](https://github.com/rariyama) ([#8997](https://github.com/dbt-labs/dbt-core/issues/8997))
+- [@scottgigante,nevdelap](https://github.com/scottgigante,nevdelap) ([#8774](https://github.com/dbt-labs/dbt-core/issues/8774))
+- [@ttusing](https://github.com/ttusing) ([#10434](https://github.com/dbt-labs/dbt-core/issues/10434))

.changes/1.9.0-b2.md

@@ -0,0 +1 @@
+## dbt-core 1.9.0-b2 - October 07, 2024

.changes/1.9.0-b3.md

@@ -0,0 +1,38 @@
+## dbt-core 1.9.0-b3 - October 30, 2024
+
+### Features
+
+- Add `order_by` and `limit` fields to saved queries. ([#10531](https://github.com/dbt-labs/dbt-core/issues/10531))
+- Enable specification of dbt_valid_to for current records ([#10187](https://github.com/dbt-labs/dbt-core/issues/10187))
+- Enable use of multi-column unique key in snapshots ([#9992](https://github.com/dbt-labs/dbt-core/issues/9992))
+- Ensure `--event-time-start` is before `--event-time-end` ([#10786](https://github.com/dbt-labs/dbt-core/issues/10786))
+- Ensure microbatch models use same `current_time` value ([#10819](https://github.com/dbt-labs/dbt-core/issues/10819))
+- Emit warning when microbatch model has no input with `event_time` config ([#10926](https://github.com/dbt-labs/dbt-core/issues/10926))
+
+### Fixes
+
+- Pass test user config to adapter pre_hook by explicitly adding test builder config to node ([#10484](https://github.com/dbt-labs/dbt-core/issues/10484))
+- Handle edge cases when a specified `--event-time-end` is equivalent to the batch size truncated batch start time ([#10824](https://github.com/dbt-labs/dbt-core/issues/10824))
+- Begin tracking execution time of microbatch model batches ([#10825](https://github.com/dbt-labs/dbt-core/issues/10825))
+- Allow instances of generic data tests to be documented ([#2578](https://github.com/dbt-labs/dbt-core/issues/2578))
+- Fix warnings for models referring to a deprecated model ([#10833](https://github.com/dbt-labs/dbt-core/issues/10833))
+- Change `lookback` default from `0` to `1` to ensure better data completeness ([#10867](https://github.com/dbt-labs/dbt-core/issues/10867))
+- Make `--event-time-start` and `--event-time-end` mutually required ([#10874](https://github.com/dbt-labs/dbt-core/issues/10874))
+- Exclude hook result from results in on-run-end context ([#7387](https://github.com/dbt-labs/dbt-core/issues/7387))
+- Implement partial parsing for all-yaml snapshots ([#10903](https://github.com/dbt-labs/dbt-core/issues/10903))
+- Restore source quoting behaviour when quoting config provided in dbt_project.yml ([#10892](https://github.com/dbt-labs/dbt-core/issues/10892))
+- Fix bug when referencing deprecated models ([#10915](https://github.com/dbt-labs/dbt-core/issues/10915))
+- Fix 'model' jinja context variable type to dict  ([#10927](https://github.com/dbt-labs/dbt-core/issues/10927))
+- Take `end_time` for batches to the ceiling to handle edge case where `event_time` column is a date ([#10868](https://github.com/dbt-labs/dbt-core/issues/10868))
+
+### Under the Hood
+
+- Remove support and testing for Python 3.8, which is now EOL. ([#10861](https://github.com/dbt-labs/dbt-core/issues/10861))
+
+### Dependencies
+
+- Bump minimnum allowed dbt-adapters version to 1.8.0 ([#N/A](https://github.com/dbt-labs/dbt-core/issues/N/A))
+
+### Contributors
+- [@danlsn](https://github.com/danlsn) ([#10915](https://github.com/dbt-labs/dbt-core/issues/10915))
+- [@plypaul](https://github.com/plypaul) ([#10531](https://github.com/dbt-labs/dbt-core/issues/10531))

.changes/1.9.0-b4.md

@@ -0,0 +1,19 @@
+## dbt-core 1.9.0-b4 - November 06, 2024
+
+### Features
+
+- Emit debug logging event whenever artifacts are written ([#10937](https://github.com/dbt-labs/dbt-core/issues/10937))
+- Support --empty for snapshots ([#10372](https://github.com/dbt-labs/dbt-core/issues/10372))
+
+### Fixes
+
+- Ensure KeyboardInterrupt/SystemExit halts microbatch model execution ([#10862](https://github.com/dbt-labs/dbt-core/issues/10862))
+- Handle exceptions in `get_execution_status` more broadly to better ensure `run_results.json` gets written ([#10934](https://github.com/dbt-labs/dbt-core/issues/10934))
+
+### Under the Hood
+
+- Behavior change for mf timespine without yaml configuration ([#10959](https://github.com/dbt-labs/dbt-core/issues/10959))
+- Behavior change for cumulative metric type param ([#10960](https://github.com/dbt-labs/dbt-core/issues/10960))
+
+### Contributors
+- [@DevonFulcher](https://github.com/DevonFulcher) ([#10959](https://github.com/dbt-labs/dbt-core/issues/10959), [#10960](https://github.com/dbt-labs/dbt-core/issues/10960))

.changes/1.9.0/Dependencies-20241030-134526.yaml

@@ -0,0 +1,6 @@
+kind: Dependencies
+body: Bump minimnum allowed dbt-adapters version to 1.8.0
+time: 2024-10-30T13:45:26.144328-05:00
+custom:
+  Author: QMalcolm
+  Issue: N/A

.changes/1.9.0/Features-20240806-144859.yaml

@@ -0,0 +1,6 @@
+kind: Features
+body: Add `order_by` and `limit` fields to saved queries.
+time: 2024-08-06T14:48:59.035914-07:00
+custom:
+  Author: plypaul
+  Issue: "10531"

.changes/1.9.0/Features-20240918-162959.yaml

@@ -0,0 +1,6 @@
+kind: Features
+body: Create 'skip_nodes_if_on_run_start_fails' behavior change flag
+time: 2024-09-18T16:29:59.268422+01:00
+custom:
+    Author: aranke
+    Issue: "7387"

.changes/1.9.0/Features-20240923-155335.yaml

@@ -0,0 +1,6 @@
+kind: Features
+body: Track incremental_strategy as part of model_run tracking event
+time: 2024-09-23T15:53:35.898928+01:00
+custom:
+  Author: michelleark
+  Issue: "10761"

.changes/1.9.0/Features-20240925-120855.yaml

@@ -0,0 +1,6 @@
+kind: Features
+body: Enable specification of dbt_valid_to for current records
+time: 2024-09-25T12:08:55.926848-04:00
+custom:
+  Author: gshank
+  Issue: "10187"