Skip to content

Commit

Permalink
Add semantic model test to test_contracts_graph_parsed.py (#8654)
Browse files Browse the repository at this point in the history
* Add semantic model test to `test_contracts_graph_parsed.py`

The tests in `test_contracts_graph_parsed.py` are meant to ensure
that we can go from objects to dictionaries and back without any
changes. We've had a desire to simplify these tests. Most tests in
this file have three to four fixtures, this test only has one. What
a test of this format ensures is that parsing a SemanticModel from
a dictionary doesn't add/drop any keys from the dictionary and that
when going back to the dictionary no keys are dropped. This style of
test will still break whenever the semantic model (or sub objects)
change. However now when that happens, only one fixture will have to
be updated (whereas previously we had to update 3-4 fixtures).

* Begin using hypothesis package for symmetry testing

Hypothesis is a python package for doing property testing. The `@given`
parameterizes a test, with it generating the arguements it has following
`strategies`. The main strategies we use is `builds` this takes in a callable
passes any sub strategies for named arguements, and will try to infer any
other arguments if the callable is typed. I found that even though the
test was run many many times, some of the `SemanticModel` properties
weren't being changed. For instance `dimensions`, `entities`, and `measures`
were always empty lists. Because of this I defined sub strategies for
some attributes of `SemanticModel`s.

* Update unittest readme to have details on test_contracts_graph_parsed methodology
  • Loading branch information
QMalcolm authored Oct 9, 2023
1 parent bb249d6 commit 70b2e15
Show file tree
Hide file tree
Showing 3 changed files with 39 additions and 0 deletions.
1 change: 1 addition & 0 deletions dev-requirements.txt
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@ docutils
flake8
flaky
freezegun==0.3.12
hypothesis
ipdb
mypy==1.4.1
pip-tools
Expand Down
17 changes: 17 additions & 0 deletions tests/unit/README.md
Original file line number Diff line number Diff line change
@@ -1 +1,18 @@
# Unit test README

## test_contracts_graph_parsed.py

### The Why
We need to ensure that we can go from objects to dictionaries and back without any
changes. If some property or property value of an object gets dropped, added, or modified
while transitioning between its different possible representations, that is problematic.

### The How
The easiest way to ensure things don't get droped, added, or modified is by starting
with an object, dictifying it, moving back to an object, and then asserting that everything
is equivalent. There are many potential edge cases though: what about optional fields, what
about lists of things, and etc. To address this we use hypothesis, which will build multiple
versions of the object we're interested in testing, and run the different generated versions
of the object through the test. This gives us confidence that for any allowable configuration
of an object, state is not changed when moving back and forth betweeen the python object
version and the seralized version.
21 changes: 21 additions & 0 deletions tests/unit/test_contracts_graph_parsed.py
Original file line number Diff line number Diff line change
@@ -1,6 +1,9 @@
import pickle
import pytest

from hypothesis import given
from hypothesis.strategies import builds, lists

from dbt.node_types import NodeType, AccessType
from dbt.contracts.files import FileHash
from dbt.contracts.graph.model_config import (
Expand Down Expand Up @@ -34,7 +37,10 @@
HookNode,
Owner,
TestMetadata,
SemanticModel,
RefArgs,
)
from dbt.contracts.graph.semantic_models import Dimension, Entity, Measure
from dbt.contracts.graph.unparsed import (
ExposureType,
FreshnessThreshold,
Expand Down Expand Up @@ -2354,3 +2360,18 @@ def basic_parsed_metric_object():
meta={},
tags=[],
)


@given(
builds(
SemanticModel,
depends_on=builds(DependsOn),
dimensions=lists(builds(Dimension)),
entities=lists(builds(Entity)),
measures=lists(builds(Measure)),
refs=lists(builds(RefArgs)),
)
)
def test_semantic_model_symmetry(semantic_model: SemanticModel):
assert semantic_model == SemanticModel.from_dict(semantic_model.to_dict())
assert semantic_model == pickle.loads(pickle.dumps(semantic_model))

0 comments on commit 70b2e15

Please sign in to comment.