Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix "dbt found two resources" error with multiple snapshot blocks in one file #4773

Merged
merged 3 commits into from
Feb 25, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,7 @@ Contributors:
### Fixes
- Fix bug causing empty node level meta, snapshot config errors ([#4459](https://github.com/dbt-labs/dbt-core/issues/4459), [#4726](https://github.com/dbt-labs/dbt-core/pull/4726))
- Fix slow `dbt run` when using Postgres adapter, by deduplicating relations in `postgres_get_relations` ([#3058](https://github.com/dbt-labs/dbt-core/issues/3058), [#4521](https://github.com/dbt-labs/dbt-core/pull/4521))
- Fix partial parsing bug with multiple snapshot blocks ([#4771](https//github.com/dbt-labs/dbt-core/issues/4772), [#4773](https://github.com/dbt-labs/dbt-core/pull/4773))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree that this one is worth backporting + including in 1.0.4


## dbt-core 1.0.3 (TBD)

Expand Down
10 changes: 5 additions & 5 deletions core/dbt/parser/partial.py
Original file line number Diff line number Diff line change
Expand Up @@ -291,10 +291,10 @@ def update_mssat_in_saved(self, new_source_file, old_source_file):
if self.already_scheduled_for_parsing(old_source_file):
return

# These files only have one node.
unique_id = None
# These files only have one node except for snapshots
unique_ids = []
if old_source_file.nodes:
unique_id = old_source_file.nodes[0]
unique_ids = old_source_file.nodes
else:
# It's not clear when this would actually happen.
# Logging in case there are other associated errors.
Expand All @@ -305,7 +305,7 @@ def update_mssat_in_saved(self, new_source_file, old_source_file):
self.deleted_manifest.files[file_id] = old_source_file
self.saved_files[file_id] = deepcopy(new_source_file)
self.add_to_pp_files(new_source_file)
if unique_id:
for unique_id in unique_ids:
self.remove_node_in_saved(new_source_file, unique_id)

def remove_node_in_saved(self, source_file, unique_id):
Expand Down Expand Up @@ -379,7 +379,7 @@ def remove_mssat_file(self, source_file):
if not source_file.nodes:
fire_event(PartialParsingMissingNodes(file_id=source_file.file_id))
return
# There is generally only 1 node for SQL files, except for macros
# There is generally only 1 node for SQL files, except for macros and snapshots
for unique_id in source_file.nodes:
self.remove_node_in_saved(source_file, unique_id)
self.schedule_referencing_nodes_for_parsing(unique_id)
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -13,3 +13,17 @@ select * from {{ ref('orders') }}

{% endsnapshot %}

{% snapshot orders2_snapshot %}

{{
config(
target_schema=schema,
strategy='check',
unique_key='id',
check_cols=['order_date'],
)
}}

select * from {{ ref('orders') }}

{% endsnapshot %}
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
- add a comment
{% snapshot orders_snapshot %}

{{
Expand All @@ -8,7 +9,22 @@
check_cols=['status'],
)
}}

select * from {{ ref('orders') }}

{% endsnapshot %}

{% snapshot orders2_snapshot %}

{{
config(
target_schema=schema,
strategy='check',
unique_key='id',
check_cols=['order_date'],
)
}}

select * from {{ ref('orders') }}

{% endsnapshot %}
Original file line number Diff line number Diff line change
Expand Up @@ -503,10 +503,12 @@ def test_postgres_pp_snapshots(self):
manifest = get_manifest()
snapshot_id = 'snapshot.test.orders_snapshot'
self.assertIn(snapshot_id, manifest.nodes)
snapshot2_id = 'snapshot.test.orders2_snapshot'
self.assertIn(snapshot2_id, manifest.nodes)

# run snapshot
results = self.run_dbt(["--partial-parse", "snapshot"])
self.assertEqual(len(results), 1)
self.assertEqual(len(results), 2)

# modify snapshot
self.copy_file('test-files/snapshot2.sql', 'snapshots/snapshot.sql')
Expand Down