Skip to content

Commit

Permalink
Begin using Mergeable supplied by dbt-common (#9509)
Browse files Browse the repository at this point in the history
* Begin using `Mergeable` supplied by `dbt-common`

We're currently in the process of moving the "data resource" portion on nodes
to `dbt/artifacts`. Some of those artifacts depend on `Mergeable` which has
been defined on core. In order to move the data resources to `dbt/artifacts`,
we thus need to move `Mergeable` upstream of core. We moved `Mergeable` to
[dbt-common](https://github.com/dbt-labs/dbt-common) in
dbt-labs/dbt-common#59, and released this change in
[dbt-common 0.1.3](https://pypi.org/project/dbt-common/0.1.3/). As such as, in
order to unblock some of the `dbt/artifacts` migration work, we first need to
update references to `Mergeable` in core to use the `dbt-common` definition.

NOTE: We include changing over to `Replaceable` from `dbt-common` in this
commit. This is because there wasn't a clean way to do it. If I moved the imports
of `Replaceable` on in the files where we updated `Mergeable` then we would
have left `Replaceable` in an inbetween state. If we had moved all instances
of `Replaceable`, it'd be out of scope for the change. As such, it makes more
sense to do that as a separate changeset.

* Remove definition of `Mergeable` from dbt/contracts/util

Although we've removed the definition of `Mergeable` we've ensured the
import paths are still available. We do this because this is under
`contracts`, and the sudden disappearance from the import path might
cause issues for community members using dbt-core as a library.

Ideally we'd define a `Mergeable` class here that inherits the
`dbt-common` definition and raise a deprecation warning on instantiation.
However, we don't have an established strategy to do so.
  • Loading branch information
QMalcolm authored Feb 2, 2024
1 parent ef03ea2 commit 93f1bd5
Show file tree
Hide file tree
Showing 4 changed files with 13 additions and 23 deletions.
6 changes: 6 additions & 0 deletions .changes/unreleased/Under the Hood-20240201-125416.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
kind: Under the Hood
body: Start using `Mergeable` from dbt-common
time: 2024-02-01T12:54:16.462414-08:00
custom:
Author: QMalcolm
Issue: "9505"
6 changes: 2 additions & 4 deletions core/dbt/contracts/graph/unparsed.py
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@
AdditionalPropertiesAllowed,
AdditionalPropertiesMixin,
)
from dbt_common.contracts.util import Mergeable
from dbt_common.exceptions import DbtInternalError, CompilationError
from dbt_common.dataclass_schema import (
dbtClassMixin,
Expand All @@ -22,10 +23,7 @@
MaturityType,
MeasureAggregationParameters,
)
from dbt.contracts.util import (
Mergeable,
Replaceable,
)
from dbt.contracts.util import Replaceable

# trigger the PathEncoder
import dbt_common.helper_types # noqa:F401
Expand Down
3 changes: 2 additions & 1 deletion core/dbt/contracts/project.py
Original file line number Diff line number Diff line change
@@ -1,7 +1,8 @@
from dbt import deprecations
from dbt.contracts.util import Replaceable, Mergeable, list_str, Identifier
from dbt.contracts.util import Replaceable, list_str, Identifier
from dbt.adapters.contracts.connection import QueryComment
from dbt_common.helper_types import NoValue
from dbt_common.contracts.util import Mergeable
from dbt_common.dataclass_schema import (
dbtClassMixin,
ValidationError,
Expand Down
21 changes: 3 additions & 18 deletions core/dbt/contracts/util.py
Original file line number Diff line number Diff line change
@@ -1,8 +1,9 @@
import dataclasses
from typing import List, Any, Tuple

from dbt_common.dataclass_schema import ValidatedStringMixin, ValidationError
from dbt_common.contracts.util import Replaceable

# Leave imports of `Mergeable` and `Replaceable` to preserve import paths
from dbt_common.contracts.util import Mergeable, Replaceable # noqa:F401


SourceKey = Tuple[str, str]
Expand All @@ -24,22 +25,6 @@ class Foo:
return []


class Mergeable(Replaceable):
def merged(self, *args):
"""Perform a shallow merge, where the last non-None write wins. This is
intended to merge dataclasses that are a collection of optional values.
"""
replacements = {}
cls = type(self)
for arg in args:
for field in dataclasses.fields(cls):
value = getattr(arg, field.name)
if value is not None:
replacements[field.name] = value

return self.replace(**replacements)


class Identifier(ValidatedStringMixin):
"""Our definition of a valid Identifier is the same as what's valid for an unquoted database table name.
Expand Down

0 comments on commit 93f1bd5

Please sign in to comment.