diff --git a/latest/getting_started/index.html b/latest/getting_started/index.html index 01513d44..381d0fe4 100644 --- a/latest/getting_started/index.html +++ b/latest/getting_started/index.html @@ -735,17 +735,31 @@

Running as an executable using

GitHub Actions#

Run dbt-bouncer as part of your CI pipeline: -

steps:
-    ...
-
-    - uses: godatadriven/dbt-bouncer@vX.X
-      with:
-        config-file: ./<PATH_TO_CONFIG_FILE>
-        output-file: results.json # optional, default does not save a results file
-        send-pr-comment: true # optional, defaults to true
-        verbose: false # optional, defaults to false
-
-    ...
+
name: CI pipeline
+
+on:
+  pull_request:
+      branches:
+          - main
+
+jobs:
+    run-dbt-bouncer:
+        permissions:
+            pull-requests: write # Required to write a comment on the PR
+        runs-on: ubuntu-latest
+        steps:
+            - name: Checkout
+              uses: actions/checkout@v4
+
+            - name: Generate or fetch dbt artifacts
+              run: ...
+
+            - uses: godatadriven/dbt-bouncer@vX.X
+              with:
+                config-file: ./<PATH_TO_CONFIG_FILE>
+                output-file: results.json # optional, default does not save a results file
+                send-pr-comment: true # optional, defaults to true
+                verbose: false # optional, defaults to false
 

We recommend pinning both a major and minor version number.

Docker#

@@ -857,7 +871,7 @@

How to add a custom check to < - September 8, 2024 + September 18, 2024 diff --git a/latest/search/search_index.json b/latest/search/search_index.json index eb88db42..dadc0964 100644 --- a/latest/search/search_index.json +++ b/latest/search/search_index.json @@ -1 +1 @@ -{"config":{"lang":["en"],"separator":"[\\s\\-]+","pipeline":["stopWordFilter"]},"docs":[{"location":"","title":"Home","text":""},{"location":"#welcome-to-dbt-bouncer","title":"Welcome to dbt-bouncer","text":"

dbt-bouncer is an open-source tool that allows you to configure and enforce conventions for your dbt project. The conventions are run against dbt's artifact files (think ./target/manifest.json) resulting in speedy tests. Conventions can be specified in a .yml file, allowing maximum customisation to the conventions you wish to follow (or create \ud83d\ude00).

Check out our Getting Started guide.

"},{"location":"#terminology","title":"Terminology","text":"
  • Check: A check is a rule run against a dbt artifact.
  • Config file: A .yml file that specifies which checks to run along with any parameters.
  • dbt artifacts directory: The directory that contains the dbt artifacts (manifest.json, etc.), generally this is ./target.
"},{"location":"#aims","title":"Aims","text":"

dbt-bouncer aims to:

  • Provide a 100% configurable way to enforce conventions in a dbt project.
  • Be as fast as possible, running checks against dbt artifacts.
  • Be as easy as possible to use, with a simple config file written in YML.
  • Be as flexible as possible, allowing checks to be written in python.
  • Provide immediate feedback when run as part of a CI pipeline.
"},{"location":"#about","title":"About","text":"

dbt-bouncer is free software, released under the MIT license. It originated at Xebia Data in Amsterdam, Netherlands. Source code is available on GitHub.

All contributions, in the form of bug reports, pull requests, feedback or discussion are welcome. See the contributing guide for more information.

"},{"location":"CONTRIBUTING/","title":"Contributing to dbt-bouncer","text":"

dbt-bouncer is open source software. Whether you are a seasoned open source contributor or a first-time committer, we welcome and encourage you to contribute code, documentation, ideas, or problem statements to this project.

"},{"location":"CONTRIBUTING/#about-this-document","title":"About this document","text":"

There are many ways to contribute to the ongoing development of dbt-bouncer, such as by participating in discussions and issues.

The rest of this document serves as a more granular guide for contributing code changes to dbt-bouncer (this repository). It is not intended as a guide for using dbt-bouncer, and some pieces assume a level of familiarity with Python development (virtualenvs, Poetry, etc). Specific code snippets in this guide assume you are using macOS or Linux and are comfortable with the command line.

If you get stuck, we're happy to help! Just open an issue or draft PR and we'll do our best to help out.

"},{"location":"CONTRIBUTING/#note","title":"Note","text":"
  • Branches: All pull requests from community contributors should target the main branch (default).
"},{"location":"CONTRIBUTING/#getting-the-code","title":"Getting the code","text":""},{"location":"CONTRIBUTING/#installing-git","title":"Installing git","text":"

You will need git in order to download and modify the dbt-bouncer source code. On macOS, the best way to download git is to just install Xcode.

"},{"location":"CONTRIBUTING/#contributors","title":"Contributors","text":"

You can contribute to dbt-bouncer by forking the dbt-bouncer repository. For a detailed overview on forking, check out the GitHub docs on forking. In short, you will need to:

  1. Fork the dbt-bouncer repository.
  2. Clone your fork locally.
  3. Check out a new branch for your proposed changes.
  4. Push changes to your fork.
  5. Open a pull request against godatadriven/dbt-bouncer from your forked repository.
"},{"location":"CONTRIBUTING/#setting-up-an-environment","title":"Setting up an environment","text":"

There are some tools that will be helpful to you in developing locally. While this is the list relevant for dbt-bouncer development, many of these tools are used commonly across open-source python projects.

"},{"location":"CONTRIBUTING/#tools","title":"Tools","text":"

These are the tools used in dbt-bouncer development and testing:

  • click to create our CLI interface.
  • GitHub Actions for automating tests and checks, once a PR is pushed to the dbt-bouncer repository.
  • make to run multiple setup or test steps in combination.
  • mypy for static type checking.
  • Poetry to manage our python virtual environment.
  • pre-commit to easily run those checks.
  • Pydantic to validate our configuration file.
  • pytest to define, discover, and run tests.
  • Ruff to lint and format python code.

A deep understanding of these tools in not required to effectively contribute to dbt-bouncer, but we recommend checking out the attached documentation if you're interested in learning more about each one.

"},{"location":"CONTRIBUTING/#virtual-environments","title":"Virtual environments","text":"

We strongly recommend using virtual environments when developing code in dbt-bouncer. We recommend creating this virtualenv in the root of the dbt-bouncer repository. To create a new virtualenv, run:

poetry shell\n

This will create a new Python virtual environment.

"},{"location":"CONTRIBUTING/#setting-environment-variables","title":"Setting environment variables","text":"

Set required environment variables by copying .env.example to .env and updating the values.

"},{"location":"CONTRIBUTING/#running-dbt-bouncer-in-development","title":"Running dbt-bouncer in development","text":""},{"location":"CONTRIBUTING/#installation","title":"Installation","text":"

First make sure that you set up your virtualenv as described in Setting up an environment. Next, install dbt-bouncer, its dependencies and pre-commit:

poetry install\npoetry run pre-commit install\n

When installed in this way, any changes you make to your local copy of the source code will be reflected immediately in your next dbt-bouncer run.

"},{"location":"CONTRIBUTING/#running-dbt-bouncer","title":"Running dbt-bouncer","text":"

With your virtualenv activated, the dbt-bouncer script should point back to the source code you've cloned on your machine. You can verify this by running which dbt-bouncer. This command should show you a path to an executable in your virtualenv. You can run dbt-bouncer using the provided example configuration file via:

poetry run dbt-bouncer --config-file dbt-bouncer-example.yml\n
"},{"location":"CONTRIBUTING/#testing","title":"Testing","text":"

Once you're able to manually test that your code change is working as expected, it's important to run existing automated tests, as well as adding some new ones. These tests will ensure that: - Your code changes do not unexpectedly break other established functionality - Your code changes can handle all known edge cases - The functionality you're adding will keep working in the future

"},{"location":"CONTRIBUTING/#note_1","title":"Note","text":"
  • Generating dbt artifacts: If you change the configuration of the dbt project located in dbt_project then you will need to re-generate the dbt artifacts used in testing. To do so, run:
make build-artifacts\n
"},{"location":"CONTRIBUTING/#test-commands","title":"Test commands","text":"

There are a few methods for running tests locally.

"},{"location":"CONTRIBUTING/#makefile","title":"makefile","text":"

There are multiple targets in the makefile to run common test suites, most notably:

# Runs unit tests\nmake test-unit\n\n# Runs integration tests\nmake test-integration\n\n# Runs all tests\nmake test\n
"},{"location":"CONTRIBUTING/#pre-commit","title":"pre-commit","text":"

pre-commit takes care of running all code-checks for formatting and linting. Run poetry run pre-commit install to install pre-commit in your local environment. Once this is done you can use the git pre-commit hooks to ensure proper formatting and linting.

"},{"location":"CONTRIBUTING/#pytest","title":"pytest","text":"

Finally, you can also run a specific test or group of tests using pytest directly. With a virtualenv active and dev dependencies installed you can do things like:

# run all unit tests in a file\npoetry run pytest ./tests/unit/checks/catalog/test_columns.py\n\n# run a specific unit test\npoetry run pytest ./tests/unit/checks/catalog/test_columns.py::test_check_columns_are_documented_in_public_models\n

See pytest usage docs for an overview of useful command-line options.

"},{"location":"CONTRIBUTING/#assorted-development-tips","title":"Assorted development tips","text":"
  • Append # type: ignore to the end of a line if you need to disable mypy on that line, preferably with the specific rule to ignore such as # type: ignore[union-attr].
"},{"location":"CONTRIBUTING/#adding-a-new-check","title":"Adding a new check","text":"

To add a new check follow the below steps:

  1. In ./src/dbt_bouncer/checks choose the appropriate directory for your check. For example, if your check only requires the manifest.json then use the manifest directory, if your check requires the catalog.json then use the catalog directory.
  2. Within the chosen directory assess if a suitable file already exists. For example, if your check applies to a model then manifest/check_models.py is a suitable location.
  3. Within the chosen file, add a Pydantic model, this object must meet the following criteria:

    • Start with \"Check\".
    • Inherit from dbt_bouncer.check_base.BaseCheck.
    • Have a name attribute that is a string whose value is the snake case equivalent of the class name.
    • A default value provided for optional input arguments and arguments that are received at execution time.
    • Have a doc string that includes a description of the check, a list of possible input parameters and at least one example.
    • A clear message in the event of a failure.
  4. After the check is added, add the check to dbt-bouncer-example.yml and run dbt-bouncer --config-file dbt-bouncer-example.yml to ensure the check succeeds.

  5. (Optional) If the dbt project located in ./dbt_project needs to be updated then do so and also run make build-artifacts to generate the new test artifacts.
  6. Add at least one happy path and one unhappy path test to ./tests. The appropriate test file will be the one matching the directory of the check. For example, if the check is in ./src/dbt_bouncer/checks/catalog/check_columns.py then the test file will be ./tests/unit/checks/catalog/test_columns.py.
  7. Run make test to ensure the tests pass.
  8. Open a PR \ud83c\udf89!
"},{"location":"CONTRIBUTING/#submitting-a-pull-request","title":"Submitting a Pull Request","text":"

Code can be merged into the current development branch main by opening a pull request. If the proposal looks like it's on the right track, then a dbt-bouncer maintainer will review the PR. They may suggest code revision for style or clarity, or request that you add unit or integration test(s). These are good things! We believe that, with a little bit of help, anyone can contribute high-quality code. Once merged, your contribution will be available for the next release of dbt-bouncer.

Automated tests run via GitHub Actions. If you're a first-time contributor, all tests will require a maintainer to approve.

Once all tests are passing and your PR has been approved, a dbt-bouncer maintainer will merge your changes into the active development branch. And that's it! Happy developing :tada:

"},{"location":"cli/","title":"CLI","text":"

This page provides documentation for the dbt-bouncer CLI.

"},{"location":"cli/#dbt-bouncer","title":"dbt-bouncer","text":"

Entrypoint for dbt-bouncer.

Raises: RuntimeError: If output file has an invalid extension.

Usage:

dbt-bouncer [OPTIONS]\n

Options:

  --config-file PUREPATH  Location of the YML config file.\n  --output-file PATH      Location of the json file where check metadata will\n                          be saved.\n  -v, --verbosity         Verbosity.\n  --version               Show the version and exit.\n  --help                  Show this message and exit.\n
"},{"location":"cli/#exit-codes","title":"Exit codes","text":"

dbt-bouncer returns the following exit codes:

  • 0: All checks have succeeded.

  • 1:

    • At least one check has failed. Check the logs for more information.
    • A fatal error occurred during check setup or check execution. Check the logs for more information.
"},{"location":"config_file/","title":"Config file","text":"

dbt-bouncer requires a config file which determines what checks are run. The following options are available, in order of priority:

  1. A file passed via the --config-file CLI flag.
  2. A file named dbt-bouncer.yml in the current working directory.
  3. A [tool.dbt-bouncer] section in pyproject.toml.

Here is an example config file in yaml:

# [Optional] Directory where the dbt artifacts exists, generally the `target` directory inside a dbt project. Defaults to `./target`.\ndbt_artifacts_dir: target\n\nmanifest_checks:\n  - name: check_macro_name_matches_file_name\n  - name: check_model_names\n    include: ^models/staging\n    model_name_pattern: ^stg_\n

And the same config in toml:

[tool.dbt-bouncer]\n# [Optional] Directory where the dbt artifacts exists, generally the `target` directory inside a dbt project. Defaults to `./target`.\ndbt_artifacts_dir = \"target\"\n\n[[tool.dbt-bouncer.manifest_checks]]\nname = \"check_macro_name_matches_file_name\"\n\n[[tool.dbt-bouncer.manifest_checks]]\nname = \"check_model_names\"\ninclude = \"^models/staging\"\nmodel_name_pattern = \"^stg_\"\n

For more example config files, see here.

"},{"location":"config_file/#common-arguments","title":"Common arguments","text":""},{"location":"config_file/#exclude-and-include","title":"Exclude and Include","text":"

Most (but not all) checks accept the following optional arguments:

  • exclude: Regexp to match which original file paths to exclude.
  • include: Regexp to match which original file paths to include.

Example per resource type:

  • Exposures: The original file path to the properties file where the source is defined, e.g. ^models/marts/finance will match exposures defined in ./models/marts/finance/_exposures.yml.
  • Macros: The original file path to the macro file, e.g. ^macros/system will match files like ./macros/system/generate_schema_name.sql.
  • Models: The original file path to the model file, e.g. ^marts will match files like ./models/marts/customers.sql.
  • Run results: The original file path to the file associated with the resource, e.g. ^seeds/finance will match seeds in ./seeds/finance, ^models/staging will match models and tests in ./models/staging.
  • Semantic models: The original file path to the properties file where the semantic model is defined, e.g. ^models/marts/finance will match semantic models defined in ./models/marts/finance/_finance__semantic_models.yml.
  • Sources: The original file path to the properties file where the source is defined, e.g. ^models/staging/crm will match sources defined in ./models/staging/crm/_crm__sources.yml.
  • Unit tests: The original file path to the properties file where the unit test is defined, e.g. ^models/staging/crm will match unit tests defined in ^staging/crm/_stg_crm__unit_tests.yml.

To determine if a check accepts these arguments view the Checks page.

Note

exclude and include can be specified at both the check level and the global level. Should both levels be specified, then the check level is applied. All the below examples result in the check_model_names check being run on all models in ./models/staging:

# Specify `include` at the check level only\nmanifest_checks:\n  - name: check_model_names\n    include: ^models/staging\n    model_name_pattern: ^stg_\n
# Specify `include` at the check and global levels\ninclude: ^models/marts\nmanifest_checks:\n  - name: check_model_names\n    include: ^models/staging\n    model_name_pattern: ^stg_\n
# Specify `include` at the global level only\ninclude: ^models/staging\nmanifest_checks:\n  - name: check_model_names\n    model_name_pattern: ^stg_\n

Note

When compiled on Windows machines, keys such as original_file_path, patch_path and path take the form:

models\\\\staging\\\\crm\\\\model_1.sql\n

When compiled on Linux and Mac machines, these same keys take the form:

models/staging/crm/model_1.sql\n

dbt-bouncer converts all of these paths to the Linux/Mac form, hence when you are supplying values to exclude and include you should use the Linux/Mac form.

"},{"location":"config_file/#severity","title":"Severity","text":"

All checks accept a severity argument, valid values are:

  • error: If the check fails then dbt-bouncer will return a non-zero exit code.
  • warn: If the check fails then dbt-bouncer will return a non-zero exit code.

severity can also be specified globally, this is useful when applying dbt-bouncer to a pre-existing dbt project. It allows you to run dbt-bouncer, identify the checks that fail and address the failures in your own time without receiving non-zero exit codes:

# Specify `severity` at the global levels: all checks will have a `warn` severity, avoiding non-zero exit codes.\nseverity: warn\n\nmanifest_checks:\n  - name: check_exposure_based_on_view\n  ...\n

Note

severity can be specified at both the check level and the global level. Should both levels be specified, then the global level is applied.

# No `severity` specified: check will have an `error` severity.\nmanifest_checks:\n  - name: check_exposure_based_on_view\n
# Specify `severity` at the check level only: check will have a `warn` severity.\nmanifest_checks:\n  - name: check_exposure_based_on_view\n    severity: warn\n
# Specify `severity` at the check and global levels: check will have a `warn` severity.\nseverity: warn\nmanifest_checks:\n  - name: check_exposure_based_on_view\n    severity: error\n
# Specify `severity` at the global level only: check will have a `warn` severity.\nseverity: warn\nmanifest_checks:\n  - name: check_exposure_based_on_view\n
"},{"location":"faq/","title":"Frequently Asked Questions","text":""},{"location":"faq/#can-other-tools-perform-the-same-checks-as-dbt-bouncer","title":"Can other tools perform the same checks as dbt-bouncer?","text":"

There are several other tools that perform similar tasks as dbt-bouncer.

  • dbt-checkpoint: A collection of pre-commit hooks for dbt projects. Tests are written in python. Configuration is performed via .pre-commit-config.yaml. Provided the dbt artifacts have already been generated, dbt-checkpoint does not need access to the underlying database. The hooks execute when a new commit is made, as such dbt-checkpoint is designed to be run only as part of pre-commit.
  • dbt-project-evaluator: This is a dbt package from dbt Labs. Tests are written in .sql files using a combination of Jinja and SQL. Configuration is performed via dbt_project.yml and seed files (i.e. csv files). Requires a connection to underlying database. Designed to be run both in a CI pipeline and also during active development.
  • dbt-score: This is a python package installable via pip. A collection of tests that apply only to dbt models. Tests can be executed from the command line. Tests are written in python. Configuration is performed via a pyproject.toml file. Provided the dbt artifacts have already been generated, dbt-score does not need access to the underlying database. Designed to be run during development.

While the above tools inhabit the same space as dbt-bouncer they do not provide what we consider to be the optimum experience that dbt-bouncer provides:

  • Designed to run both locally and in a CI pipeline.
  • Configurable via a file format, YML, that dbt developers are already familiar with.
  • Does not require database access.
  • Can run tests against any of dbt's artifacts.
  • Allows tests to be written in python.

As such we consider dbt-bouncer to be the best tool to enforce conventions in a dbt project.

Tip

dbt-bouncer can perform all the tests currently included in dbt-checkpoint, dbt-project-evaluator and dbt-score. If you see an existing test that is not possible with dbt-bouncer, open an issue and we'll add it!

"},{"location":"faq/#how-to-set-up-dbt-bouncer-in-a-monorepo","title":"How to set up dbt-bouncer in a monorepo?","text":"

A monorepo may consist of one directory with a dbt project and other directories with unrelated code. It may be desired for dbt-bouncer to be configured from the root directory. Sample directory tree:

.\n\u251c\u2500\u2500 dbt-bouncer.yml\n\u251c\u2500\u2500 README.md\n\u251c\u2500\u2500 dbt-project\n\u2502   \u251c\u2500\u2500 models\n\u2502   \u251c\u2500\u2500 dbt_project.yml\n\u2502   \u2514\u2500\u2500 profiles.yml\n\u2514\u2500\u2500 package-a\n    \u251c\u2500\u2500 src\n    \u251c\u2500\u2500 tests\n    \u2514\u2500\u2500 package.json\n

To ease configuration you can use exclude or include at the global level (see Config File for more details). For the above example dbt-bouncer.yml could be configured as:

dbt_artifacts_dir: dbt-project/target\ninclude: ^dbt-project\n\nmanifest_checks:\n    - name: check_exposure_based_on_non_public_models\n

dbt-bouncer can now be run from the root directory.

"},{"location":"getting_started/","title":"Getting Started","text":""},{"location":"getting_started/#how-to-run-dbt-bouncer","title":"How to run dbt-bouncer","text":"
  1. Generate dbt artifacts by running a dbt command:

    • dbt parse to generate a manifest.json artifact.
    • dbt docs generate to generate a catalog.json artifact (necessary if you are using catalog checks).
    • dbt run (or any other command that implies it e.g. dbt build) to generate a run_results.json artifact (necessary if you are using run results checks).
  2. Create a dbt-bouncer.yml config file, details here.

  3. Run dbt-bouncer to validate that your conventions are being maintained.

"},{"location":"getting_started/#installing-with-python","title":"Installing with Python","text":"

Install from pypi.org:

pip install dbt-bouncer # or via any other package manager\n

Run:

dbt-bouncer --config-file <PATH_TO_CONFIG_FILE>\n
Running dbt-bouncer (X.X.X)...\nLoaded config from dbt-bouncer-example.yml...\nValidating conf...\n

dbt-bouncer also supports a verbose mode, run:

dbt-bouncer --config-file <PATH_TO_CONFIG_FILE> -v\n
Running dbt-bouncer (X.X.X)...\nconfig_file=PosixPath('dbt-bouncer-example.yml')\nconfig_file_source='COMMANDLINE'\nConfig file passed via command line: dbt-bouncer-example.yml\nLoading config from /home/pslattery/repos/dbt-bouncer/dbt-bouncer-example.yml...\nLoading config from dbt-bouncer-example.yml...\nLoaded config from dbt-bouncer-example.yml...\nconf={'dbt_artifacts_dir': 'dbt_project/target', 'catalog_checks': [{'name': 'check_column_name_complies_to_column_type', 'column_name_pattern': '^is_.*', 'exclude': '^staging', 'types': ['BOOLEAN']}]}\nValidating conf...\n
"},{"location":"getting_started/#running-as-an-executable-using-uv","title":"Running as an executable using uv","text":"

Run dbt-bouncer as a standalone Python executable using uv:

uvx dbt-bouncer --config-file <PATH_TO_CONFIG_FILE>\n
"},{"location":"getting_started/#github-actions","title":"GitHub Actions","text":"

Run dbt-bouncer as part of your CI pipeline:

steps:\n    ...\n\n    - uses: godatadriven/dbt-bouncer@vX.X\n      with:\n        config-file: ./<PATH_TO_CONFIG_FILE>\n        output-file: results.json # optional, default does not save a results file\n        send-pr-comment: true # optional, defaults to true\n        verbose: false # optional, defaults to false\n\n    ...\n

We recommend pinning both a major and minor version number.

"},{"location":"getting_started/#docker","title":"Docker","text":"

Run dbt-bouncer via Docker:

docker run --rm \\\n    --volume \"$PWD\":/app \\\n    ghcr.io/godatadriven/dbt-bouncer:vX.X.X \\\n    --config-file /app/<PATH_TO_CONFIG_FILE>\n
"},{"location":"getting_started/#pex","title":"Pex","text":"

You can also run the .pex (Python EXecutable) artifact directly once you have a python executable (3.8 -> 3.12) installed:

wget https://github.com/godatadriven/dbt-bouncer/releases/download/vX.X.X/dbt-bouncer.pex -O dbt-bouncer.pex\n\npython dbt-bouncer.pex --config-file $PWD/<PATH_TO_CONFIG_FILE>\n
"},{"location":"getting_started/#how-to-contribute-a-check-to-dbt-bouncer","title":"How to contribute a check to dbt-bouncer","text":"

See Adding a new check.

"},{"location":"getting_started/#how-to-add-a-custom-check-to-dbt-bouncer","title":"How to add a custom check to dbt-bouncer","text":"

In addition to the checks built into dbt-bouncer, the ability to add custom checks is supported. This allows users to write checks that are specific to the conventions of their projects. To add a custom check:

  1. Create an empty directory and add a custom_checks_dir key to your config file. The value of this key should be the path to the directory you just created, relative to where the config file is located.
  2. In this directory create an empty __init__.py file.
  3. In this directory create a subdirectory named catalog, manifest or run_results depending on the type of artifact you want to check.
  4. In this subdirectory create a python file that defines a check. The check must meet the following criteria:

    • Start with \"Check\".
    • Inherit from dbt_bouncer.check_base.BaseCheck.
    • Have a name attribute that is a string whose value is the snake case equivalent of the class name.
    • A default value provided for optional input arguments and arguments that are received at execution time.
    • Have a doc string that includes a description of the check, a list of possible input parameters and at least one example.
    • A clear message in the event of a failure.
  5. In your config file, add the name of the check and any desired arguments.

  6. Run dbt-bouncer, your custom check will be executed.

An example:

  • Directory tree:

    .\n\u251c\u2500\u2500 dbt-bouncer.yml\n\u251c\u2500\u2500 dbt_project.yml\n\u251c\u2500\u2500 my_custom_checks\n|   \u251c\u2500\u2500 __init__.py\n|   \u2514\u2500\u2500 manifest\n|       \u2514\u2500\u2500 check_custom_to_me.py\n\u2514\u2500\u2500 target\n    \u2514\u2500\u2500 manifest.json\n
  • Contents of check_custom_to_me.py:

    from typing import TYPE_CHECKING, Literal\n\nfrom pydantic import Field\n\nfrom dbt_bouncer.check_base import BaseCheck\n\nif TYPE_CHECKING:\n    import warnings\n\n    with warnings.catch_warnings():\n        warnings.filterwarnings(\"ignore\", category=UserWarning)\n        from dbt_bouncer.parsers import DbtBouncerModelBase\n\n\nclass CheckModelDepcrecationDate(BaseCheck):\n\n    model: \"DbtBouncerModelBase\" = Field(default=None)\n    name: Literal[\"check_model_deprecation_date\"]\n\n    def execute(self) -> None:\n        \"\"\"Execute the check.\"\"\"\n\n        assert self.model.deprecation_date is not None, f\"`{self.model.name}` requires a `deprecation_date` to be set.\"\n
  • Contents of dbt-bouncer.yml:

    custom_checks_dir: my_custom_checks\n\nmanifest_checks:\n    - name: check_model_deprecation_date\n      include: ^models/staging/legacy_erp\n
"},{"location":"checks/","title":"Checks","text":"

dbt-bouncer runs checks against artifacts from dbt. These checks fall into three categories:

  • Catalog checks:
    • Catalog Sources
    • Columns
  • Manifest checks:
    • Exposures
    • Lineage
    • Macros
    • Metadata
    • Models
    • Semantic Models
    • Sources
    • Unit Tests
  • Run Results checks:
    • Run Results
"},{"location":"checks/catalog/check_catalog_sources/","title":"Catalog Checks: Catalog Sources","text":"

Note

The below checks require both catalog.json and manifest.json to be present.

"},{"location":"checks/catalog/check_catalog_sources/#catalog.check_catalog_sources.CheckSourceColumnsAreAllDocumented","title":"CheckSourceColumnsAreAllDocumented","text":"

All columns in a source should be included in the source's properties file, i.e. .yml file.

Receives at execution time:

Name Type Description catalog_source CatalogTable

The CatalogTable object to check.

sources List[DbtBouncerSourceBase]

List of DbtBouncerSourceBase objects parsed from catalog.json.

Other Parameters (passed via config file):

Name Type Description exclude Optional[str]

Regex pattern to match the source path (i.e the .yml file where the source is configured). Source paths that match the pattern will not be checked.

include Optional[str]

Regex pattern to match the source path (i.e the .yml file where the source is configured). Only source paths that match the pattern will be checked.

severity Optional[Literal['error', 'warn']]

Severity level of the check. Default: error.

Example(s):

catalog_checks:\n    - name: check_source_columns_are_all_documented\n

Source code in src/dbt_bouncer/checks/catalog/check_catalog_sources.py
class CheckSourceColumnsAreAllDocumented(BaseCheck):\n    \"\"\"All columns in a source should be included in the source's properties file, i.e. `.yml` file.\n\n    Receives:\n        catalog_source (CatalogTable): The CatalogTable object to check.\n        sources (List[DbtBouncerSourceBase]): List of DbtBouncerSourceBase objects parsed from `catalog.json`.\n\n    Other Parameters:\n        exclude (Optional[str]): Regex pattern to match the source path (i.e the .yml file where the source is configured). Source paths that match the pattern will not be checked.\n        include (Optional[str]): Regex pattern to match the source path (i.e the .yml file where the source is configured). Only source paths that match the pattern will be checked.\n        severity (Optional[Literal[\"error\", \"warn\"]]): Severity level of the check. Default: `error`.\n\n    Example(s):\n        ```yaml\n        catalog_checks:\n            - name: check_source_columns_are_all_documented\n        ```\n\n    \"\"\"\n\n    catalog_source: \"CatalogTable\" = Field(default=None)\n    name: Literal[\"check_source_columns_are_all_documented\"]\n    sources: List[\"DbtBouncerSourceBase\"] = Field(default=[])\n\n    def execute(self) -> None:\n        \"\"\"Execute the check.\"\"\"\n        source = next(\n            s for s in self.sources if s.unique_id == self.catalog_source.unique_id\n        )\n        undocumented_columns = [\n            v.name\n            for _, v in self.catalog_source.columns.items()\n            if v.name not in source.columns\n        ]\n        assert not undocumented_columns, f\"`{self.catalog_source.unique_id}` has columns that are not included in the sources properties file: {undocumented_columns}\"\n
"},{"location":"checks/catalog/check_columns/","title":"Catalog Checks: Columns","text":"

Note

The below checks require both catalog.json and manifest.json to be present.

"},{"location":"checks/catalog/check_columns/#catalog.check_columns.CheckColumnDescriptionPopulated","title":"CheckColumnDescriptionPopulated","text":"

Columns must have a populated description.

Receives at execution time:

Name Type Description catalog_node CatalogTable

The CatalogTable object to check.

models List[DbtBouncerModelBase]

List of DbtBouncerModelBase objects parsed from manifest.json.

Other Parameters (passed via config file):

Name Type Description exclude Optional[str]

Regex pattern to match the model path. Model paths that match the pattern will not be checked.

include Optional[str]

Regex pattern to match the model path. Only model paths that match the pattern will be checked.

severity Optional[Literal['error', 'warn']]

Severity level of the check. Default: error.

Example(s):

manifest_checks:\n    - name: check_column_description_populated\n      include: ^models/marts\n

Source code in src/dbt_bouncer/checks/catalog/check_columns.py
class CheckColumnDescriptionPopulated(BaseCheck):\n    \"\"\"Columns must have a populated description.\n\n    Receives:\n        catalog_node (CatalogTable): The CatalogTable object to check.\n        models (List[DbtBouncerModelBase]): List of DbtBouncerModelBase objects parsed from `manifest.json`.\n\n    Other Parameters:\n        exclude (Optional[str]): Regex pattern to match the model path. Model paths that match the pattern will not be checked.\n        include (Optional[str]): Regex pattern to match the model path. Only model paths that match the pattern will be checked.\n        severity (Optional[Literal[\"error\", \"warn\"]]): Severity level of the check. Default: `error`.\n\n    Example(s):\n        ```yaml\n        manifest_checks:\n            - name: check_column_description_populated\n              include: ^models/marts\n        ```\n\n    \"\"\"\n\n    catalog_node: \"CatalogTable\" = Field(default=None)\n    models: List[\"DbtBouncerModelBase\"] = Field(default=[])\n    name: Literal[\"check_column_description_populated\"]\n\n    def execute(self) -> None:\n        \"\"\"Execute the check.\"\"\"\n        if self.catalog_node.unique_id.split(\".\")[0] == \"model\":\n            model = next(\n                m for m in self.models if m.unique_id == self.catalog_node.unique_id\n            )\n            non_complying_columns = []\n            for _, v in self.catalog_node.columns.items():\n                if (\n                    model.columns.get(v.name) is None\n                    or len(model.columns[v.name].description.strip()) <= 4\n                ):\n                    non_complying_columns.append(v.name)\n\n            assert not non_complying_columns, f\"`{self.catalog_node.unique_id.split('.')[-1]}` has columns that do not have a populated description: {non_complying_columns}\"\n
"},{"location":"checks/catalog/check_columns/#catalog.check_columns.CheckColumnNameCompliesToColumnType","title":"CheckColumnNameCompliesToColumnType","text":"

Columns with specified data types must comply to the specified regexp naming pattern.

Parameters:

Name Type Description Default column_name_pattern str

Regex pattern to match the model name.

required types List[str]

List of data types to check.

required

Receives at execution time:

Name Type Description catalog_node CatalogTable

The CatalogTable object to check.

Other Parameters (passed via config file):

Name Type Description exclude Optional[str]

Regex pattern to match the model path. Model paths that match the pattern will not be checked.

include Optional[str]

Regex pattern to match the model path. Only model paths that match the pattern will be checked.

severity Optional[Literal['error', 'warn']]

Severity level of the check. Default: error.

Example(s):

catalog_checks:\n    # DATE columns must end with \"_date\"\n    - name: check_column_name_complies_to_column_type\n      column_name_pattern: .*_date$\n      types:\n        - DATE\n
catalog_checks:\n    # BOOLEAN columns must start with \"is_\"\n    - name: check_column_name_complies_to_column_type\n      column_name_pattern: ^is_.*\n      types:\n        - BOOLEAN\n
catalog_checks:\n    # Columns of all types must consist of lowercase letters and underscores. Note that the specified types depend on the underlying database.\n    - name: check_column_name_complies_to_column_type\n      column_name_pattern: ^[a-z_]*$\n      types:\n        - BIGINT\n        - BOOLEAN\n        - DATE\n        - DOUBLE\n        - INTEGER\n        - VARCHAR\n

Source code in src/dbt_bouncer/checks/catalog/check_columns.py
class CheckColumnNameCompliesToColumnType(BaseCheck):\n    \"\"\"Columns with specified data types must comply to the specified regexp naming pattern.\n\n    Parameters:\n        column_name_pattern (str): Regex pattern to match the model name.\n        types (List[str]): List of data types to check.\n\n    Receives:\n        catalog_node (CatalogTable): The CatalogTable object to check.\n\n    Other Parameters:\n        exclude (Optional[str]): Regex pattern to match the model path. Model paths that match the pattern will not be checked.\n        include (Optional[str]): Regex pattern to match the model path. Only model paths that match the pattern will be checked.\n        severity (Optional[Literal[\"error\", \"warn\"]]): Severity level of the check. Default: `error`.\n\n    Example(s):\n        ```yaml\n        catalog_checks:\n            # DATE columns must end with \"_date\"\n            - name: check_column_name_complies_to_column_type\n              column_name_pattern: .*_date$\n              types:\n                - DATE\n        ```\n        ```yaml\n        catalog_checks:\n            # BOOLEAN columns must start with \"is_\"\n            - name: check_column_name_complies_to_column_type\n              column_name_pattern: ^is_.*\n              types:\n                - BOOLEAN\n        ```\n        ```yaml\n        catalog_checks:\n            # Columns of all types must consist of lowercase letters and underscores. Note that the specified types depend on the underlying database.\n            - name: check_column_name_complies_to_column_type\n              column_name_pattern: ^[a-z_]*$\n              types:\n                - BIGINT\n                - BOOLEAN\n                - DATE\n                - DOUBLE\n                - INTEGER\n                - VARCHAR\n        ```\n\n    \"\"\"\n\n    catalog_node: \"CatalogTable\" = Field(default=None)\n    column_name_pattern: str\n    name: Literal[\"check_column_name_complies_to_column_type\"]\n    types: List[str]\n\n    def execute(self) -> None:\n        \"\"\"Execute the check.\"\"\"\n        non_complying_columns = [\n            v.name\n            for _, v in self.catalog_node.columns.items()\n            if v.type in self.types\n            and re.compile(self.column_name_pattern.strip()).match(v.name) is None\n        ]\n\n        assert not non_complying_columns, f\"`{self.catalog_node.unique_id.split('.')[-1]}` has columns that don't comply with the specified regexp pattern (`{self.column_name_pattern}`): {non_complying_columns}\"\n
"},{"location":"checks/catalog/check_columns/#catalog.check_columns.CheckColumnsAreAllDocumented","title":"CheckColumnsAreAllDocumented","text":"

All columns in a model should be included in the model's properties file, i.e. .yml file.

Receives at execution time:

Name Type Description catalog_node CatalogTable

The CatalogTable object to check.

models List[DbtBouncerModel]

List of DbtBouncerModel objects parsed from manifest.json.

Other Parameters (passed via config file):

Name Type Description exclude Optional[str]

Regex pattern to match the model path. Model paths that match the pattern will not be checked.

include Optional[str]

Regex pattern to match the model path. Only model paths that match the pattern will be checked.

severity Optional[Literal['error', 'warn']]

Severity level of the check. Default: error.

Example(s):

catalog_checks:\n    - name: check_columns_are_all_documented\n

Source code in src/dbt_bouncer/checks/catalog/check_columns.py
class CheckColumnsAreAllDocumented(BaseCheck):\n    \"\"\"All columns in a model should be included in the model's properties file, i.e. `.yml` file.\n\n    Receives:\n        catalog_node (CatalogTable): The CatalogTable object to check.\n        models (List[DbtBouncerModel]): List of DbtBouncerModel objects parsed from `manifest.json`.\n\n    Other Parameters:\n        exclude (Optional[str]): Regex pattern to match the model path. Model paths that match the pattern will not be checked.\n        include (Optional[str]): Regex pattern to match the model path. Only model paths that match the pattern will be checked.\n        severity (Optional[Literal[\"error\", \"warn\"]]): Severity level of the check. Default: `error`.\n\n    Example(s):\n        ```yaml\n        catalog_checks:\n            - name: check_columns_are_all_documented\n        ```\n\n    \"\"\"\n\n    catalog_node: \"CatalogTable\" = Field(default=None)\n    models: List[\"DbtBouncerModelBase\"] = Field(default=[])\n    name: Literal[\"check_columns_are_all_documented\"]\n\n    def execute(self) -> None:\n        \"\"\"Execute the check.\"\"\"\n        if self.catalog_node.unique_id.split(\".\")[0] == \"model\":\n            model = next(\n                m for m in self.models if m.unique_id == self.catalog_node.unique_id\n            )\n            undocumented_columns = [\n                v.name\n                for _, v in self.catalog_node.columns.items()\n                if v.name not in model.columns\n            ]\n            assert not undocumented_columns, f\"`{self.catalog_node.unique_id.split('.')[-1]}` has columns that are not included in the models properties file: {undocumented_columns}\"\n
"},{"location":"checks/catalog/check_columns/#catalog.check_columns.CheckColumnsAreDocumentedInPublicModels","title":"CheckColumnsAreDocumentedInPublicModels","text":"

Columns should have a populated description in public models.

Receives at execution time:

Name Type Description catalog_node CatalogTable

The CatalogTable object to check.

models List[DbtBouncerModel]

List of DbtBouncerModel objects parsed from manifest.json.

Other Parameters (passed via config file):

Name Type Description exclude Optional[str]

Regex pattern to match the model path. Model paths that match the pattern will not be checked.

include Optional[str]

Regex pattern to match the model path. Only model paths that match the pattern will be checked.

severity Optional[Literal['error', 'warn']]

Severity level of the check. Default: error.

Example(s):

catalog_checks:\n    - name: check_columns_are_documented_in_public_models\n

Source code in src/dbt_bouncer/checks/catalog/check_columns.py
class CheckColumnsAreDocumentedInPublicModels(BaseCheck):\n    \"\"\"Columns should have a populated description in public models.\n\n    Receives:\n        catalog_node (CatalogTable): The CatalogTable object to check.\n        models (List[DbtBouncerModel]): List of DbtBouncerModel objects parsed from `manifest.json`.\n\n    Other Parameters:\n        exclude (Optional[str]): Regex pattern to match the model path. Model paths that match the pattern will not be checked.\n        include (Optional[str]): Regex pattern to match the model path. Only model paths that match the pattern will be checked.\n        severity (Optional[Literal[\"error\", \"warn\"]]): Severity level of the check. Default: `error`.\n\n    Example(s):\n        ```yaml\n        catalog_checks:\n            - name: check_columns_are_documented_in_public_models\n        ```\n\n    \"\"\"\n\n    catalog_node: \"CatalogTable\" = Field(default=None)\n    models: List[\"DbtBouncerModelBase\"] = Field(default=[])\n    name: Literal[\"check_columns_are_documented_in_public_models\"]\n\n    def execute(self) -> None:\n        \"\"\"Execute the check.\"\"\"\n        if self.catalog_node.unique_id.split(\".\")[0] == \"model\":\n            model = next(\n                m for m in self.models if m.unique_id == self.catalog_node.unique_id\n            )\n            non_complying_columns = []\n            for _, v in self.catalog_node.columns.items():\n                if model.access.value == \"public\":\n                    column_config = model.columns.get(v.name)\n                    if (\n                        column_config is None\n                        or len(column_config.description.strip()) < 4\n                    ):\n                        non_complying_columns.append(v.name)\n\n            assert not non_complying_columns, f\"`{self.catalog_node.unique_id.split('.')[-1]}` is a public model but has columns that don't have a populated description: {non_complying_columns}\"\n
"},{"location":"checks/catalog/check_columns/#catalog.check_columns.CheckColumnHasSpecifiedTest","title":"CheckColumnHasSpecifiedTest","text":"

Columns that match the specified regexp pattern must have a specified test.

Parameters:

Name Type Description Default column_name_pattern str

Regex pattern to match the column name.

required test_name str

Name of the test to check for.

required

Receives at execution time:

Name Type Description catalog_node CatalogTable

The CatalogTable object to check.

tests List[DbtBouncerTestBase]

List of DbtBouncerTestBase objects parsed from manifest.json.

Other Parameters (passed via config file):

Name Type Description exclude Optional[str]

Regex pattern to match the model path. Model paths that match the pattern will not be checked.

include Optional[str]

Regex pattern to match the model path. Only model paths that match the pattern will be checked.

severity Optional[Literal['error', 'warn']]

Severity level of the check. Default: error.

Example(s):

catalog_checks:\n    - name: check_column_has_specified_test\n      column_name_pattern: ^is_.*\n      test_name: not_null\n

Source code in src/dbt_bouncer/checks/catalog/check_columns.py
class CheckColumnHasSpecifiedTest(BaseCheck):\n    \"\"\"Columns that match the specified regexp pattern must have a specified test.\n\n    Parameters:\n        column_name_pattern (str): Regex pattern to match the column name.\n        test_name (str): Name of the test to check for.\n\n    Receives:\n        catalog_node (CatalogTable): The CatalogTable object to check.\n        tests (List[DbtBouncerTestBase]): List of DbtBouncerTestBase objects parsed from `manifest.json`.\n\n    Other Parameters:\n        exclude (Optional[str]): Regex pattern to match the model path. Model paths that match the pattern will not be checked.\n        include (Optional[str]): Regex pattern to match the model path. Only model paths that match the pattern will be checked.\n        severity (Optional[Literal[\"error\", \"warn\"]]): Severity level of the check. Default: `error`.\n\n    Example(s):\n        ```yaml\n        catalog_checks:\n            - name: check_column_has_specified_test\n              column_name_pattern: ^is_.*\n              test_name: not_null\n        ```\n\n    \"\"\"\n\n    catalog_node: \"CatalogTable\" = Field(default=None)\n    column_name_pattern: str\n    name: Literal[\"check_column_has_specified_test\"]\n    test_name: str\n    tests: List[\"DbtBouncerTestBase\"] = Field(default=[])\n\n    def execute(self) -> None:\n        \"\"\"Execute the check.\"\"\"\n        columns_to_check = [\n            v.name\n            for _, v in self.catalog_node.columns.items()\n            if re.compile(self.column_name_pattern.strip()).match(v.name) is not None\n        ]\n        relevant_tests = [\n            t\n            for t in self.tests\n            if hasattr(t, \"test_metadata\") is True\n            and hasattr(t, \"attached_node\") is True\n            and t.test_metadata.name == self.test_name\n            and t.attached_node == self.catalog_node.unique_id\n        ]\n        non_complying_columns = [\n            c\n            for c in columns_to_check\n            if f\"{self.catalog_node.unique_id}.{c}\"\n            not in [f\"{t.attached_node}.{t.column_name}\" for t in relevant_tests]\n        ]\n\n        assert not non_complying_columns, f\"`{self.catalog_node.unique_id.split('.')[-1]}` has columns that should have a `{self.test_name}` test: {non_complying_columns}\"\n
"},{"location":"checks/manifest/check_exposures/","title":"Manifest Checks: Exposures","text":"

Note

The below checks require manifest.json to be present.

"},{"location":"checks/manifest/check_exposures/#manifest.check_exposures.CheckExposureOnNonPublicModels","title":"CheckExposureOnNonPublicModels","text":"

Exposures should be based on public models only.

Receives at execution time:

Name Type Description exposure DbtBouncerExposureBase

The DbtBouncerExposureBase object to check.

models List[DbtBouncerModelBase]

List of DbtBouncerModelBase objects parsed from manifest.json.

Other Parameters (passed via config file):

Name Type Description exclude Optional[str]

Regex pattern to match the exposure path (i.e the .yml file where the exposure is configured). Exposure paths that match the pattern will not be checked.

include Optional[str]

Regex pattern to match the exposure path (i.e the .yml file where the exposure is configured). Only exposure paths that match the pattern will be checked.

severity Optional[Literal['error', 'warn']]

Severity level of the check. Default: error.

Example(s):

manifest_checks:\n    - name: check_exposure_based_on_non_public_models\n

Source code in src/dbt_bouncer/checks/manifest/check_exposures.py
class CheckExposureOnNonPublicModels(BaseCheck):\n    \"\"\"Exposures should be based on public models only.\n\n    Receives:\n        exposure (DbtBouncerExposureBase): The DbtBouncerExposureBase object to check.\n        models (List[DbtBouncerModelBase]): List of DbtBouncerModelBase objects parsed from `manifest.json`.\n\n    Other Parameters:\n        exclude (Optional[str]): Regex pattern to match the exposure path (i.e the .yml file where the exposure is configured). Exposure paths that match the pattern will not be checked.\n        include (Optional[str]): Regex pattern to match the exposure path (i.e the .yml file where the exposure is configured). Only exposure paths that match the pattern will be checked.\n        severity (Optional[Literal[\"error\", \"warn\"]]): Severity level of the check. Default: `error`.\n\n    Example(s):\n        ```yaml\n        manifest_checks:\n            - name: check_exposure_based_on_non_public_models\n        ```\n\n    \"\"\"\n\n    exposure: \"DbtBouncerExposureBase\" = Field(default=None)\n    models: List[\"DbtBouncerModelBase\"] = Field(default=[])\n    name: Literal[\"check_exposure_based_on_non_public_models\"]\n\n    def execute(self) -> None:\n        \"\"\"Execute the check.\"\"\"\n        non_public_upstream_dependencies = []\n        for model in self.exposure.depends_on.nodes:\n            if (\n                model.split(\".\")[0] == \"model\"\n                and model.split(\".\")[1] == self.exposure.package_name\n            ):\n                model = next(m for m in self.models if m.unique_id == model)\n                if model.access.value != \"public\":\n                    non_public_upstream_dependencies.append(model.name)\n\n        assert not non_public_upstream_dependencies, f\"`{self.exposure.name}` is based on a model(s) that is not public: {non_public_upstream_dependencies}.\"\n
"},{"location":"checks/manifest/check_exposures/#manifest.check_exposures.CheckExposureOnView","title":"CheckExposureOnView","text":"

Exposures should not be based on views.

Parameters:

Name Type Description Default materializations_to_include Optional[List[str]]

List of materializations to include in the check.

required

Receives at execution time:

Name Type Description exposure DbtBouncerExposureBase

The DbtBouncerExposureBase object to check.

models List[DbtBouncerModelBase]

List of DbtBouncerModelBase objects parsed from manifest.json.

Other Parameters (passed via config file):

Name Type Description exclude Optional[str]

Regex pattern to match the exposure path (i.e the .yml file where the exposure is configured). Exposure paths that match the pattern will not be checked.

include Optional[str]

Regex pattern to match the exposure path (i.e the .yml file where the exposure is configured). Only exposure paths that match the pattern will be checked.

severity Optional[Literal['error', 'warn']]

Severity level of the check. Default: error.

Example(s):

manifest_checks:\n    - name: check_exposure_based_on_view\n
manifest_checks:\n    - name: check_exposure_based_on_view\n      materializations_to_include:\n        - ephemeral\n        - my_custom_materialization\n        - view\n

Source code in src/dbt_bouncer/checks/manifest/check_exposures.py
class CheckExposureOnView(BaseCheck):\n    \"\"\"Exposures should not be based on views.\n\n    Parameters:\n        materializations_to_include (Optional[List[str]]): List of materializations to include in the check.\n\n    Receives:\n        exposure (DbtBouncerExposureBase): The DbtBouncerExposureBase object to check.\n        models (List[DbtBouncerModelBase]): List of DbtBouncerModelBase objects parsed from `manifest.json`.\n\n    Other Parameters:\n        exclude (Optional[str]): Regex pattern to match the exposure path (i.e the .yml file where the exposure is configured). Exposure paths that match the pattern will not be checked.\n        include (Optional[str]): Regex pattern to match the exposure path (i.e the .yml file where the exposure is configured). Only exposure paths that match the pattern will be checked.\n        severity (Optional[Literal[\"error\", \"warn\"]]): Severity level of the check. Default: `error`.\n\n    Example(s):\n        ```yaml\n        manifest_checks:\n            - name: check_exposure_based_on_view\n        ```\n        ```yaml\n        manifest_checks:\n            - name: check_exposure_based_on_view\n              materializations_to_include:\n                - ephemeral\n                - my_custom_materialization\n                - view\n        ```\n\n    \"\"\"\n\n    exposure: \"DbtBouncerExposureBase\" = Field(default=None)\n    materializations_to_include: List[str] = Field(\n        default=[\"ephemeral\", \"view\"],\n    )\n    models: List[\"DbtBouncerModelBase\"] = Field(default=[])\n    name: Literal[\"check_exposure_based_on_view\"]\n\n    def execute(self) -> None:\n        \"\"\"Execute the check.\"\"\"\n        non_table_upstream_dependencies = []\n        for model in self.exposure.depends_on.nodes:\n            if (\n                model.split(\".\")[0] == \"model\"\n                and model.split(\".\")[1] == self.exposure.package_name\n            ):\n                model = next(m for m in self.models if m.unique_id == model)\n                if model.config.materialized in self.materializations_to_include:\n                    non_table_upstream_dependencies.append(model.name)\n\n        assert not non_table_upstream_dependencies, f\"`{self.exposure.name}` is based on a model that is not a table: {non_table_upstream_dependencies}.\"\n
"},{"location":"checks/manifest/check_lineage/","title":"Manifest Checks: Lineage","text":"

Note

The below checks require manifest.json to be present.

"},{"location":"checks/manifest/check_lineage/#manifest.check_lineage.CheckLineagePermittedUpstreamModels","title":"CheckLineagePermittedUpstreamModels","text":"

Upstream models must have a path that matches the provided upstream_path_pattern.

Parameters:

Name Type Description Default upstream_path_pattern str

Regexp pattern to match the upstream model(s) path.

required

Receives at execution time:

Name Type Description manifest_obj DbtBouncerManifest

The manifest object.

model DbtBouncerModelBase

The DbtBouncerModelBase object to check.

models List[DbtBouncerModelBase]

List of DbtBouncerModelBase objects parsed from manifest.json.

Other Parameters (passed via config file):

Name Type Description exclude Optional[str]

Regex pattern to match the model path. Model paths that match the pattern will not be checked.

include Optional[str]

Regex pattern to match the model path. Only model paths that match the pattern will be checked.

severity Optional[Literal['error', 'warn']]

Severity level of the check. Default: error.

Example(s):

manifest_checks:\n    - name: check_lineage_permitted_upstream_models\n      include: ^models/staging\n      upstream_path_pattern: $^\n    - name: check_lineage_permitted_upstream_models\n      include: ^models/intermediate\n      upstream_path_pattern: ^models/staging|^models/intermediate\n    - name: check_lineage_permitted_upstream_models\n      include: ^models/marts\n      upstream_path_pattern: ^models/staging|^models/intermediate\n

Source code in src/dbt_bouncer/checks/manifest/check_lineage.py
class CheckLineagePermittedUpstreamModels(BaseCheck):\n    \"\"\"Upstream models must have a path that matches the provided `upstream_path_pattern`.\n\n    Parameters:\n        upstream_path_pattern (str): Regexp pattern to match the upstream model(s) path.\n\n    Receives:\n        manifest_obj (DbtBouncerManifest): The manifest object.\n        model (DbtBouncerModelBase): The DbtBouncerModelBase object to check.\n        models (List[DbtBouncerModelBase]): List of DbtBouncerModelBase objects parsed from `manifest.json`.\n\n    Other Parameters:\n        exclude (Optional[str]): Regex pattern to match the model path. Model paths that match the pattern will not be checked.\n        include (Optional[str]): Regex pattern to match the model path. Only model paths that match the pattern will be checked.\n        severity (Optional[Literal[\"error\", \"warn\"]]): Severity level of the check. Default: `error`.\n\n    Example(s):\n        ```yaml\n        manifest_checks:\n            - name: check_lineage_permitted_upstream_models\n              include: ^models/staging\n              upstream_path_pattern: $^\n            - name: check_lineage_permitted_upstream_models\n              include: ^models/intermediate\n              upstream_path_pattern: ^models/staging|^models/intermediate\n            - name: check_lineage_permitted_upstream_models\n              include: ^models/marts\n              upstream_path_pattern: ^models/staging|^models/intermediate\n        ```\n\n    \"\"\"\n\n    manifest_obj: \"DbtBouncerManifest\" = Field(default=None)\n    model: \"DbtBouncerModelBase\" = Field(default=None)\n    models: List[\"DbtBouncerModelBase\"] = Field(default=[])\n    name: Literal[\"check_lineage_permitted_upstream_models\"]\n    upstream_path_pattern: str\n\n    def execute(self) -> None:\n        \"\"\"Execute the check.\"\"\"\n        upstream_models = [\n            x\n            for x in self.model.depends_on.nodes\n            if x.split(\".\")[0] == \"model\"\n            and x.split(\".\")[1] == self.manifest_obj.manifest.metadata.project_name\n        ]\n        not_permitted_upstream_models = [\n            upstream_model\n            for upstream_model in upstream_models\n            if re.compile(self.upstream_path_pattern.strip()).match(\n                clean_path_str(\n                    next(\n                        m for m in self.models if m.unique_id == upstream_model\n                    ).original_file_path\n                ),\n            )\n            is None\n        ]\n        assert not not_permitted_upstream_models, f\"`{self.model.name}` references upstream models that are not permitted: {[m.split('.')[-1] for m in not_permitted_upstream_models]}.\"\n
"},{"location":"checks/manifest/check_lineage/#manifest.check_lineage.CheckLineageSeedCannotBeUsed","title":"CheckLineageSeedCannotBeUsed","text":"

Seed cannot be referenced in models with a path that matches the specified include config.

Receives at execution time:

Name Type Description model DbtBouncerModelBase

The DbtBouncerModelBase object to check.

Other Parameters (passed via config file):

Name Type Description exclude Optional[str]

Regex pattern to match the model path. Model paths that match the pattern will not be checked.

include Optional[str]

Regex pattern to match the model path. Only model paths that match the pattern will be checked.

severity Optional[Literal['error', 'warn']]

Severity level of the check. Default: error.

Example(s):

manifest_checks:\n    - name: check_lineage_seed_cannot_be_used\n      include: ^models/intermediate|^models/marts\n

Source code in src/dbt_bouncer/checks/manifest/check_lineage.py
class CheckLineageSeedCannotBeUsed(BaseCheck):\n    \"\"\"Seed cannot be referenced in models with a path that matches the specified `include` config.\n\n    Receives:\n        model (DbtBouncerModelBase): The DbtBouncerModelBase object to check.\n\n    Other Parameters:\n        exclude (Optional[str]): Regex pattern to match the model path. Model paths that match the pattern will not be checked.\n        include (Optional[str]): Regex pattern to match the model path. Only model paths that match the pattern will be checked.\n        severity (Optional[Literal[\"error\", \"warn\"]]): Severity level of the check. Default: `error`.\n\n    Example(s):\n        ```yaml\n        manifest_checks:\n            - name: check_lineage_seed_cannot_be_used\n              include: ^models/intermediate|^models/marts\n        ```\n\n    \"\"\"\n\n    model: \"DbtBouncerModelBase\" = Field(default=None)\n    name: Literal[\"check_lineage_seed_cannot_be_used\"]\n\n    def execute(self) -> None:\n        \"\"\"Execute the check.\"\"\"\n        assert not [\n            x for x in self.model.depends_on.nodes if x.split(\".\")[0] == \"seed\"\n        ], f\"`{self.model.name}` references a seed even though this is not permitted.\"\n
"},{"location":"checks/manifest/check_lineage/#manifest.check_lineage.CheckLineageSourceCannotBeUsed","title":"CheckLineageSourceCannotBeUsed","text":"

Sources cannot be referenced in models with a path that matches the specified include config.

Receives at execution time:

Name Type Description model DbtBouncerModelBase

The DbtBouncerModelBase object to check.

Other Parameters (passed via config file):

Name Type Description exclude Optional[str]

Regex pattern to match the model path. Model paths that match the pattern will not be checked.

include Optional[str]

Regex pattern to match the model path. Only model paths that match the pattern will be checked.

severity Optional[Literal['error', 'warn']]

Severity level of the check. Default: error.

Example(s):

manifest_checks:\n    - name: check_lineage_source_cannot_be_used\n      include: ^models/intermediate|^models/marts\n

Source code in src/dbt_bouncer/checks/manifest/check_lineage.py
class CheckLineageSourceCannotBeUsed(BaseCheck):\n    \"\"\"Sources cannot be referenced in models with a path that matches the specified `include` config.\n\n    Receives:\n        model (DbtBouncerModelBase): The DbtBouncerModelBase object to check.\n\n    Other Parameters:\n        exclude (Optional[str]): Regex pattern to match the model path. Model paths that match the pattern will not be checked.\n        include (Optional[str]): Regex pattern to match the model path. Only model paths that match the pattern will be checked.\n        severity (Optional[Literal[\"error\", \"warn\"]]): Severity level of the check. Default: `error`.\n\n    Example(s):\n        ```yaml\n        manifest_checks:\n            - name: check_lineage_source_cannot_be_used\n              include: ^models/intermediate|^models/marts\n        ```\n\n    \"\"\"\n\n    model: \"DbtBouncerModelBase\" = Field(default=None)\n    name: Literal[\"check_lineage_source_cannot_be_used\"]\n\n    def execute(self) -> None:\n        \"\"\"Execute the check.\"\"\"\n        assert not [\n            x for x in self.model.depends_on.nodes if x.split(\".\")[0] == \"source\"\n        ], f\"`{self.model.name}` references a source even though this is not permitted.\"\n
"},{"location":"checks/manifest/check_macros/","title":"Manifest Checks: Macros","text":"

Note

The below checks require manifest.json to be present.

"},{"location":"checks/manifest/check_macros/#manifest.check_macros.CheckMacroArgumentsDescriptionPopulated","title":"CheckMacroArgumentsDescriptionPopulated","text":"

Macro arguments must have a populated description.

Receives at execution time:

Name Type Description macro Macros

The Macros object to check.

Other Parameters (passed via config file):

Name Type Description exclude Optional[str]

Regex pattern to match the macro path. Macro paths that match the pattern will not be checked.

include Optional[str]

Regex pattern to match the macro path. Only macro paths that match the pattern will be checked.

severity Optional[Literal['error', 'warn']]

Severity level of the check. Default: error.

Example(s):

manifest_checks:\n    - name: check_macro_arguments_description_populated\n
# Only \"common\" macros need to have their arguments populated\nmanifest_checks:\n    - name: check_macro_arguments_description_populated\n      include: ^macros/common\n

Source code in src/dbt_bouncer/checks/manifest/check_macros.py
class CheckMacroArgumentsDescriptionPopulated(BaseCheck):\n    \"\"\"Macro arguments must have a populated description.\n\n    Receives:\n        macro (Macros): The Macros object to check.\n\n    Other Parameters:\n        exclude (Optional[str]): Regex pattern to match the macro path. Macro paths that match the pattern will not be checked.\n        include (Optional[str]): Regex pattern to match the macro path. Only macro paths that match the pattern will be checked.\n        severity (Optional[Literal[\"error\", \"warn\"]]): Severity level of the check. Default: `error`.\n\n    Example(s):\n        ```yaml\n        manifest_checks:\n            - name: check_macro_arguments_description_populated\n        ```\n        ```yaml\n        # Only \"common\" macros need to have their arguments populated\n        manifest_checks:\n            - name: check_macro_arguments_description_populated\n              include: ^macros/common\n        ```\n\n    \"\"\"\n\n    macro: \"Macros\" = Field(default=None)\n    name: Literal[\"check_macro_arguments_description_populated\"]\n\n    def execute(self) -> None:\n        \"\"\"Execute the check.\"\"\"\n        environment = jinja2.Environment(autoescape=True, extensions=[TagExtension])\n        ast = environment.parse(self.macro.macro_sql)\n\n        if hasattr(ast.body[0], \"args\"):\n            # Assume macro is a \"true\" macro\n            macro_arguments = [a.name for a in ast.body[0].args]\n        else:\n            if \"materialization\" in [\n                x.value.value\n                for x in ast.body[0].nodes[0].kwargs  # type: ignore[attr-defined]\n                if isinstance(x.value, jinja2.nodes.Const)\n            ]:\n                # Materializations don't have arguments\n                macro_arguments = []\n            else:\n                # Macro is a test\n                test_macro = next(\n                    x\n                    for x in ast.body\n                    if not isinstance(x.nodes[0], jinja2.nodes.Call)  # type: ignore[attr-defined]\n                )\n                macro_arguments = [\n                    x.name\n                    for x in test_macro.nodes  # type: ignore[attr-defined]\n                    if isinstance(x, jinja2.nodes.Name)\n                ]\n\n        # macro_arguments: List of args parsed from macro SQL\n        # macro.arguments: List of args manually added to the properties file\n\n        non_complying_args = []\n        for arg in macro_arguments:\n            macro_doc_raw = [x for x in self.macro.arguments if x.name == arg]\n            if macro_doc_raw == [] or (\n                arg not in [x.name for x in self.macro.arguments]\n                or len(macro_doc_raw[0].description.strip()) <= 4\n            ):\n                non_complying_args.append(arg)\n\n        assert (\n            non_complying_args == []\n        ), f\"Macro `{self.macro.name}` does not have a populated description for the following argument(s): {non_complying_args}.\"\n
"},{"location":"checks/manifest/check_macros/#manifest.check_macros.CheckMacroCodeDoesNotContainRegexpPattern","title":"CheckMacroCodeDoesNotContainRegexpPattern","text":"

The raw code for a macro must not match the specified regexp pattern.

Parameters:

Name Type Description Default regexp_pattern str

The regexp pattern that should not be matched by the macro code.

required

Receives at execution time:

Name Type Description macro Macros

The Macros object to check.

Other Parameters (passed via config file):

Name Type Description exclude Optional[str]

Regex pattern to match the macro path. Macro paths that match the pattern will not be checked.

include Optional[str]

Regex pattern to match the macro path. Only macro paths that match the pattern will be checked.

severity Optional[Literal['error', 'warn']]

Severity level of the check. Default: error.

Example(s):

manifest_checks:\n    # Prefer `coalesce` over `ifnull`: https://docs.sqlfluff.com/en/stable/rules.html#sqlfluff.rules.sphinx.Rule_CV02\n    - name: check_macro_code_does_not_contain_regexp_pattern\n      regexp_pattern: .*[i][f][n][u][l][l].*\n

Source code in src/dbt_bouncer/checks/manifest/check_macros.py
class CheckMacroCodeDoesNotContainRegexpPattern(BaseCheck):\n    \"\"\"The raw code for a macro must not match the specified regexp pattern.\n\n    Parameters:\n        regexp_pattern (str): The regexp pattern that should not be matched by the macro code.\n\n    Receives:\n        macro (Macros): The Macros object to check.\n\n    Other Parameters:\n        exclude (Optional[str]): Regex pattern to match the macro path. Macro paths that match the pattern will not be checked.\n        include (Optional[str]): Regex pattern to match the macro path. Only macro paths that match the pattern will be checked.\n        severity (Optional[Literal[\"error\", \"warn\"]]): Severity level of the check. Default: `error`.\n\n    Example(s):\n        ```yaml\n        manifest_checks:\n            # Prefer `coalesce` over `ifnull`: https://docs.sqlfluff.com/en/stable/rules.html#sqlfluff.rules.sphinx.Rule_CV02\n            - name: check_macro_code_does_not_contain_regexp_pattern\n              regexp_pattern: .*[i][f][n][u][l][l].*\n        ```\n\n    \"\"\"\n\n    macro: \"Macros\" = Field(default=None)\n    name: Literal[\"check_macro_code_does_not_contain_regexp_pattern\"]\n    regexp_pattern: str\n\n    def execute(self) -> None:\n        \"\"\"Execute the check.\"\"\"\n        assert (\n            re.compile(self.regexp_pattern.strip(), flags=re.DOTALL).match(\n                self.macro.macro_sql\n            )\n            is None\n        ), f\"Macro `{self.macro.name}` contains a banned string: `{self.regexp_pattern.strip()}`.\"\n
"},{"location":"checks/manifest/check_macros/#manifest.check_macros.CheckMacroDescriptionPopulated","title":"CheckMacroDescriptionPopulated","text":"

Macros must have a populated description.

Receives at execution time:

Name Type Description macro Macros

The Macros object to check.

Other Parameters (passed via config file):

Name Type Description exclude Optional[str]

Regex pattern to match the macro path. Macro paths that match the pattern will not be checked.

include Optional[str]

Regex pattern to match the macro path. Only macro paths that match the pattern will be checked.

severity Optional[Literal['error', 'warn']]

Severity level of the check. Default: error.

Example(s):

manifest_checks:\n    - name: check_macro_description_populated\n
# Only \"common\" macros need to have a populated description\nmanifest_checks:\n    - name: check_macro_description_populated\n      include: ^macros/common\n

Source code in src/dbt_bouncer/checks/manifest/check_macros.py
class CheckMacroDescriptionPopulated(BaseCheck):\n    \"\"\"Macros must have a populated description.\n\n    Receives:\n        macro (Macros): The Macros object to check.\n\n    Other Parameters:\n        exclude (Optional[str]): Regex pattern to match the macro path. Macro paths that match the pattern will not be checked.\n        include (Optional[str]): Regex pattern to match the macro path. Only macro paths that match the pattern will be checked.\n        severity (Optional[Literal[\"error\", \"warn\"]]): Severity level of the check. Default: `error`.\n\n    Example(s):\n        ```yaml\n        manifest_checks:\n            - name: check_macro_description_populated\n        ```\n        ```yaml\n        # Only \"common\" macros need to have a populated description\n        manifest_checks:\n            - name: check_macro_description_populated\n              include: ^macros/common\n        ```\n\n    \"\"\"\n\n    macro: \"Macros\" = Field(default=None)\n    name: Literal[\"check_macro_description_populated\"]\n\n    def execute(self) -> None:\n        \"\"\"Execute the check.\"\"\"\n        assert (\n            len(self.macro.description.strip()) > 4\n        ), f\"Macro `{self.macro.name}` does not have a populated description.\"\n
"},{"location":"checks/manifest/check_macros/#manifest.check_macros.CheckMacroMaxNumberOfLines","title":"CheckMacroMaxNumberOfLines","text":"

Macros may not have more than the specified number of lines.

Parameters:

Name Type Description Default max_number_of_lines int

The maximum number of permitted lines.

required

Receives at execution time:

Name Type Description macro Macros

The Macros object to check.

Other Parameters (passed via config file):

Name Type Description exclude Optional[str]

Regex pattern to match the macro path. Macro paths that match the pattern will not be checked.

include Optional[str]

Regex pattern to match the macro path. Only macro paths that match the pattern will be checked.

severity Optional[Literal['error', 'warn']]

Severity level of the check. Default: error.

Example(s):

manifest_checks:\n    - name: check_macro_max_number_of_lines\n
manifest_checks:\n    - name: check_macro_max_number_of_lines\n      max_number_of_lines: 100\n

Source code in src/dbt_bouncer/checks/manifest/check_macros.py
class CheckMacroMaxNumberOfLines(BaseCheck):\n    \"\"\"Macros may not have more than the specified number of lines.\n\n    Parameters:\n        max_number_of_lines (int): The maximum number of permitted lines.\n\n    Receives:\n        macro (Macros): The Macros object to check.\n\n    Other Parameters:\n        exclude (Optional[str]): Regex pattern to match the macro path. Macro paths that match the pattern will not be checked.\n        include (Optional[str]): Regex pattern to match the macro path. Only macro paths that match the pattern will be checked.\n        severity (Optional[Literal[\"error\", \"warn\"]]): Severity level of the check. Default: `error`.\n\n    Example(s):\n        ```yaml\n        manifest_checks:\n            - name: check_macro_max_number_of_lines\n        ```\n        ```yaml\n        manifest_checks:\n            - name: check_macro_max_number_of_lines\n              max_number_of_lines: 100\n        ```\n\n    \"\"\"\n\n    macro: \"Macros\" = Field(default=None)\n    name: Literal[\"check_macro_max_number_of_lines\"]\n    max_number_of_lines: int = Field(default=50)\n\n    def execute(self) -> None:\n        \"\"\"Execute the check.\"\"\"\n        actual_number_of_lines = self.macro.macro_sql.count(\"\\n\") + 1\n\n        assert (\n            actual_number_of_lines <= self.max_number_of_lines\n        ), f\"Macro `{self.macro.name}` has {actual_number_of_lines} lines, this is more than the maximum permitted number of lines ({self.max_number_of_lines}).\"\n
"},{"location":"checks/manifest/check_macros/#manifest.check_macros.CheckMacroNameMatchesFileName","title":"CheckMacroNameMatchesFileName","text":"

Macros names must be the same as the file they are contained in.

Generic tests are also macros, however to document these tests the \"name\" value must be preceded with \"test_\".

Receives at execution time:

Name Type Description macro Macros

The Macros object to check.

Other Parameters (passed via config file):

Name Type Description exclude Optional[str]

Regex pattern to match the macro path. Macro paths that match the pattern will not be checked.

include Optional[str]

Regex pattern to match the macro path. Only macro paths that match the pattern will be checked.

severity Optional[Literal['error', 'warn']]

Severity level of the check. Default: error.

Example(s):

manifest_checks:\n    - name: check_macro_name_matches_file_name\n

Source code in src/dbt_bouncer/checks/manifest/check_macros.py
class CheckMacroNameMatchesFileName(BaseCheck):\n    \"\"\"Macros names must be the same as the file they are contained in.\n\n    Generic tests are also macros, however to document these tests the \"name\" value must be preceded with \"test_\".\n\n    Receives:\n        macro (Macros): The Macros object to check.\n\n    Other Parameters:\n        exclude (Optional[str]): Regex pattern to match the macro path. Macro paths that match the pattern will not be checked.\n        include (Optional[str]): Regex pattern to match the macro path. Only macro paths that match the pattern will be checked.\n        severity (Optional[Literal[\"error\", \"warn\"]]): Severity level of the check. Default: `error`.\n\n    Example(s):\n        ```yaml\n        manifest_checks:\n            - name: check_macro_name_matches_file_name\n        ```\n\n    \"\"\"\n\n    macro: \"Macros\" = Field(default=None)\n    name: Literal[\"check_macro_name_matches_file_name\"]\n\n    def execute(self) -> None:\n        \"\"\"Execute the check.\"\"\"\n        if self.macro.name.startswith(\"test_\"):\n            assert (\n                self.macro.name[5:]\n                == clean_path_str(self.macro.original_file_path)\n                .split(\"/\")[-1]\n                .split(\".\")[0]\n            ), f\"Macro `{self.macro.unique_id}` is not in a file named `{self.macro.name[5:]}.sql`.\"\n        else:\n            assert (\n                self.macro.name\n                == clean_path_str(self.macro.original_file_path)\n                .split(\"/\")[-1]\n                .split(\".\")[0]\n            ), f\"Macro `{self.macro.name}` is not in a file of the same name.\"\n
"},{"location":"checks/manifest/check_macros/#manifest.check_macros.CheckMacroPropertyFileLocation","title":"CheckMacroPropertyFileLocation","text":"

Macro properties files must follow the guidance provided by dbt here.

Receives at execution time:

Name Type Description macro Macros

The Macros object to check.

Other Parameters (passed via config file):

Name Type Description exclude Optional[str]

Regex pattern to match the macro path. Macro paths that match the pattern will not be checked.

include Optional[str]

Regex pattern to match the macro path. Only macro paths that match the pattern will be checked.

severity Optional[Literal['error', 'warn']]

Severity level of the check. Default: error.

Example(s):

manifest_checks:\n    - name: check_macro_property_file_location\n

Source code in src/dbt_bouncer/checks/manifest/check_macros.py
class CheckMacroPropertyFileLocation(BaseCheck):\n    \"\"\"Macro properties files must follow the guidance provided by dbt [here](https://docs.getdbt.com/best-practices/how-we-structure/5-the-rest-of-the-project#how-we-use-the-other-folders).\n\n    Receives:\n        macro (Macros): The Macros object to check.\n\n    Other Parameters:\n        exclude (Optional[str]): Regex pattern to match the macro path. Macro paths that match the pattern will not be checked.\n        include (Optional[str]): Regex pattern to match the macro path. Only macro paths that match the pattern will be checked.\n        severity (Optional[Literal[\"error\", \"warn\"]]): Severity level of the check. Default: `error`.\n\n    Example(s):\n        ```yaml\n        manifest_checks:\n            - name: check_macro_property_file_location\n        ```\n\n    \"\"\"\n\n    macro: \"Macros\" = Field(default=None)\n    name: Literal[\"check_macro_property_file_location\"]\n\n    def execute(self) -> None:\n        \"\"\"Execute the check.\"\"\"\n        expected_substr = \"_\".join(\n            clean_path_str(self.macro.original_file_path)[6:].split(\"/\")[:-1]\n        )\n\n        assert (\n            clean_path_str(self.macro.patch_path) is not None\n        ), f\"Macro `{self.macro.name}` is not defined in a `.yml` properties file.\"\n        properties_yml_name = clean_path_str(self.macro.patch_path).split(\"/\")[-1]\n\n        if clean_path_str(self.macro.original_file_path).startswith(\n            \"tests/\",\n        ):  # Do not check generic tests (which are also macros)\n            pass\n        elif expected_substr == \"\":  # i.e. macro in ./macros\n            assert (\n                properties_yml_name == \"_macros.yml\"\n            ), f\"The properties file for `{self.macro.name}` (`{properties_yml_name}`) should be `_macros.yml`.\"\n        else:\n            assert properties_yml_name.startswith(\n                \"_\",\n            ), f\"The properties file for `{self.macro.name}` (`{properties_yml_name}`) does not start with an underscore.\"\n            assert (\n                expected_substr in properties_yml_name\n            ), f\"The properties file for `{self.macro.name}` (`{properties_yml_name}`) does not contain the expected substring (`{expected_substr}`).\"\n            assert properties_yml_name.endswith(\n                \"__macros.yml\",\n            ), f\"The properties file for `{self.macro.name.name}` (`{properties_yml_name}`) does not end with `__macros.yml`.\"\n
"},{"location":"checks/manifest/check_metadata/","title":"Manifest Checks: Metadata","text":"

Note

The below checks require manifest.json to be present.

"},{"location":"checks/manifest/check_metadata/#manifest.check_metadata.CheckProjectName","title":"CheckProjectName","text":"

Enforce that the name of the dbt project matches a supplied regex. Generally used to enforce that project names conform to something like company_<DOMAIN>.

Parameters:

Name Type Description Default project_name_pattern str

Regex pattern to match the project name.

required

Receives at execution time:

Name Type Description manifest_obj DbtBouncerManifest

The manifest object.

Other Parameters (passed via config file):

Name Type Description severity Optional[Literal['error', 'warn']]

Severity level of the check. Default: error.

Example(s):

manifest_checks:\n    - name: check_project_name\n      project_name_pattern: ^awesome_company_\n

Source code in src/dbt_bouncer/checks/manifest/check_metadata.py
class CheckProjectName(BaseModel):\n    \"\"\"Enforce that the name of the dbt project matches a supplied regex. Generally used to enforce that project names conform to something like  `company_<DOMAIN>`.\n\n    Parameters:\n        project_name_pattern (str): Regex pattern to match the project name.\n\n    Receives:\n        manifest_obj (DbtBouncerManifest): The manifest object.\n\n    Other Parameters:\n        severity (Optional[Literal[\"error\", \"warn\"]]): Severity level of the check. Default: `error`.\n\n    Example(s):\n        ```yaml\n        manifest_checks:\n            - name: check_project_name\n              project_name_pattern: ^awesome_company_\n        ```\n\n    \"\"\"\n\n    model_config = ConfigDict(extra=\"forbid\")\n\n    index: Optional[int] = Field(\n        default=None,\n        description=\"Index to uniquely identify the check, calculated at runtime.\",\n    )\n    manifest_obj: \"DbtBouncerManifest\" = Field(default=None)\n    name: Literal[\"check_project_name\"]\n    project_name_pattern: str\n    severity: Optional[Literal[\"error\", \"warn\"]] = Field(\n        default=\"error\",\n        description=\"Severity of the check, one of 'error' or 'warn'.\",\n    )\n\n    def execute(self) -> None:\n        \"\"\"Execute the check.\"\"\"\n        assert (\n            re.compile(self.project_name_pattern.strip()).match(\n                self.manifest_obj.manifest.metadata.project_name,\n            )\n            is not None\n        ), f\"Project name (`{self.manifest_obj.manifest.metadata.project_name}`) does not conform to the supplied regex `({self.project_name_pattern.strip()})`.\"\n
"},{"location":"checks/manifest/check_models/","title":"Manifest Checks: Models","text":"

Note

The below checks require manifest.json to be present.

"},{"location":"checks/manifest/check_models/#manifest.check_models.CheckModelAccess","title":"CheckModelAccess","text":"

Models must have the specified access attribute. Requires dbt 1.7+.

Parameters:

Name Type Description Default access Literal['private', 'protected', 'public']

The access level to check for.

required

Receives at execution time:

Name Type Description model DbtBouncerModelBase

The DbtBouncerModelBase object to check.

Other Parameters (passed via config file):

Name Type Description exclude Optional[str]

Regex pattern to match the model path. Model paths that match the pattern will not be checked.

include Optional[str]

Regex pattern to match the model path. Only model paths that match the pattern will be checked.

severity Optional[Literal['error', 'warn']]

Severity level of the check. Default: error.

Example(s):

manifest_checks:\n    # Align with dbt best practices that marts should be `public`, everything else should be `protected`\n    - name: check_model_access\n      access: protected\n      include: ^models/intermediate\n    - name: check_model_access\n      access: public\n      include: ^models/marts\n    - name: check_model_access\n      access: protected\n      include: ^models/staging\n

Source code in src/dbt_bouncer/checks/manifest/check_models.py
class CheckModelAccess(BaseCheck):\n    \"\"\"Models must have the specified access attribute. Requires dbt 1.7+.\n\n    Parameters:\n        access (Literal[\"private\", \"protected\", \"public\"]): The access level to check for.\n\n    Receives:\n        model (DbtBouncerModelBase): The DbtBouncerModelBase object to check.\n\n    Other Parameters:\n        exclude (Optional[str]): Regex pattern to match the model path. Model paths that match the pattern will not be checked.\n        include (Optional[str]): Regex pattern to match the model path. Only model paths that match the pattern will be checked.\n        severity (Optional[Literal[\"error\", \"warn\"]]): Severity level of the check. Default: `error`.\n\n    Example(s):\n        ```yaml\n        manifest_checks:\n            # Align with dbt best practices that marts should be `public`, everything else should be `protected`\n            - name: check_model_access\n              access: protected\n              include: ^models/intermediate\n            - name: check_model_access\n              access: public\n              include: ^models/marts\n            - name: check_model_access\n              access: protected\n              include: ^models/staging\n        ```\n\n    \"\"\"\n\n    access: Literal[\"private\", \"protected\", \"public\"]\n    model: \"DbtBouncerModelBase\" = Field(default=None)\n    name: Literal[\"check_model_access\"]\n\n    def execute(self) -> None:\n        \"\"\"Execute the check.\"\"\"\n        assert (\n            self.model.access.value == self.access\n        ), f\"`{self.model.name}` has `{self.model.access.value}` access, it should have access `{self.access}`.\"\n
"},{"location":"checks/manifest/check_models/#manifest.check_models.CheckModelContractsEnforcedForPublicModel","title":"CheckModelContractsEnforcedForPublicModel","text":"

Public models must have contracts enforced.

Receives at execution time:

Name Type Description model DbtBouncerModelBase

The DbtBouncerModelBase object to check.

Other Parameters (passed via config file):

Name Type Description exclude Optional[str]

Regex pattern to match the model path. Model paths that match the pattern will not be checked.

include Optional[str]

Regex pattern to match the model path. Only model paths that match the pattern will be checked.

severity Optional[Literal['error', 'warn']]

Severity level of the check. Default: error.

Example(s):

manifest_checks:\n    - name: check_model_contract_enforced_for_public_model\n

Source code in src/dbt_bouncer/checks/manifest/check_models.py
class CheckModelContractsEnforcedForPublicModel(BaseCheck):\n    \"\"\"Public models must have contracts enforced.\n\n    Receives:\n        model (DbtBouncerModelBase): The DbtBouncerModelBase object to check.\n\n    Other Parameters:\n        exclude (Optional[str]): Regex pattern to match the model path. Model paths that match the pattern will not be checked.\n        include (Optional[str]): Regex pattern to match the model path. Only model paths that match the pattern will be checked.\n        severity (Optional[Literal[\"error\", \"warn\"]]): Severity level of the check. Default: `error`.\n\n    Example(s):\n        ```yaml\n        manifest_checks:\n            - name: check_model_contract_enforced_for_public_model\n        ```\n\n    \"\"\"\n\n    model: \"DbtBouncerModelBase\" = Field(default=None)\n    name: Literal[\"check_model_contract_enforced_for_public_model\"]\n\n    def execute(self) -> None:\n        \"\"\"Execute the check.\"\"\"\n        if self.model.access.value == \"public\":\n            assert (\n                self.model.contract.enforced is True\n            ), f\"`{self.model.name}` is a public model but does not have contracts enforced.\"\n
"},{"location":"checks/manifest/check_models/#manifest.check_models.CheckModelDescriptionPopulated","title":"CheckModelDescriptionPopulated","text":"

Models must have a populated description.

Receives at execution time:

Name Type Description model DbtBouncerModelBase

The DbtBouncerModelBase object to check.

Other Parameters (passed via config file):

Name Type Description exclude Optional[str]

Regex pattern to match the model path. Model paths that match the pattern will not be checked.

include Optional[str]

Regex pattern to match the model path. Only model paths that match the pattern will be checked.

severity Optional[Literal['error', 'warn']]

Severity level of the check. Default: error.

Example(s):

manifest_checks:\n    - name: check_model_description_populated\n

Source code in src/dbt_bouncer/checks/manifest/check_models.py
class CheckModelDescriptionPopulated(BaseCheck):\n    \"\"\"Models must have a populated description.\n\n    Receives:\n        model (DbtBouncerModelBase): The DbtBouncerModelBase object to check.\n\n    Other Parameters:\n        exclude (Optional[str]): Regex pattern to match the model path. Model paths that match the pattern will not be checked.\n        include (Optional[str]): Regex pattern to match the model path. Only model paths that match the pattern will be checked.\n        severity (Optional[Literal[\"error\", \"warn\"]]): Severity level of the check. Default: `error`.\n\n    Example(s):\n        ```yaml\n        manifest_checks:\n            - name: check_model_description_populated\n        ```\n\n    \"\"\"\n\n    model: \"DbtBouncerModelBase\" = Field(default=None)\n    name: Literal[\"check_model_description_populated\"]\n\n    def execute(self) -> None:\n        \"\"\"Execute the check.\"\"\"\n        assert (\n            len(self.model.description.strip()) > 4\n        ), f\"`{self.model.name}` does not have a populated description.\"\n
"},{"location":"checks/manifest/check_models/#manifest.check_models.CheckModelsDocumentationCoverage","title":"CheckModelsDocumentationCoverage","text":"

Set the minimum percentage of models that have a populated description.

Parameters:

Name Type Description Default min_model_documentation_coverage_pct float

The minimum percentage of models that must have a populated description.

required

Receives at execution time:

Name Type Description models List[DbtBouncerModelBase]

List of DbtBouncerModelBase objects parsed from manifest.json.

Other Parameters (passed via config file):

Name Type Description severity Optional[Literal['error', 'warn']]

Severity level of the check. Default: error.

Example(s):

manifest_checks:\n    - name: check_model_description_populated\n      min_model_documentation_coverage_pct: 90\n

Source code in src/dbt_bouncer/checks/manifest/check_models.py
class CheckModelsDocumentationCoverage(BaseModel):\n    \"\"\"Set the minimum percentage of models that have a populated description.\n\n    Parameters:\n        min_model_documentation_coverage_pct (float): The minimum percentage of models that must have a populated description.\n\n    Receives:\n        models (List[DbtBouncerModelBase]): List of DbtBouncerModelBase objects parsed from `manifest.json`.\n\n    Other Parameters:\n        severity (Optional[Literal[\"error\", \"warn\"]]): Severity level of the check. Default: `error`.\n\n    Example(s):\n        ```yaml\n        manifest_checks:\n            - name: check_model_description_populated\n              min_model_documentation_coverage_pct: 90\n        ```\n\n    \"\"\"\n\n    model_config = ConfigDict(extra=\"forbid\")\n\n    index: Optional[int] = Field(\n        default=None,\n        description=\"Index to uniquely identify the check, calculated at runtime.\",\n    )\n    min_model_documentation_coverage_pct: int = Field(\n        default=100,\n        ge=0,\n        le=100,\n    )\n    models: List[\"DbtBouncerModelBase\"] = Field(default=[])\n    name: Literal[\"check_model_documentation_coverage\"]\n    severity: Optional[Literal[\"error\", \"warn\"]] = Field(\n        default=\"error\",\n        description=\"Severity of the check, one of 'error' or 'warn'.\",\n    )\n\n    def execute(self) -> None:\n        \"\"\"Execute the check.\"\"\"\n        num_models = len(self.models)\n        models_with_description = []\n        for model in self.models:\n            if len(model.description.strip()) > 4:\n                models_with_description.append(model.unique_id)\n\n        num_models_with_descriptions = len(models_with_description)\n        model_description_coverage_pct = (\n            num_models_with_descriptions / num_models\n        ) * 100\n\n        assert (\n            model_description_coverage_pct >= self.min_model_documentation_coverage_pct\n        ), f\"Only {model_description_coverage_pct}% of models have a populated description, this is less than the permitted minimum of {self.min_model_documentation_coverage_pct}%.\"\n
"},{"location":"checks/manifest/check_models/#manifest.check_models.CheckModelDocumentedInSameDirectory","title":"CheckModelDocumentedInSameDirectory","text":"

Models must be documented in the same directory where they are defined (i.e. .yml and .sql files are in the same directory).

Receives at execution time:

Name Type Description model DbtBouncerModelBase

The DbtBouncerModelBase object to check.

Other Parameters (passed via config file):

Name Type Description exclude Optional[str]

Regex pattern to match the model path. Model paths that match the pattern will not be checked.

include Optional[str]

Regex pattern to match the model path. Only model paths that match the pattern will be checked.

severity Optional[Literal['error', 'warn']]

Severity level of the check. Default: error.

Example(s):

manifest_checks:\n    - name: check_model_documented_in_same_directory\n

Source code in src/dbt_bouncer/checks/manifest/check_models.py
class CheckModelDocumentedInSameDirectory(BaseCheck):\n    \"\"\"Models must be documented in the same directory where they are defined (i.e. `.yml` and `.sql` files are in the same directory).\n\n    Receives:\n        model (DbtBouncerModelBase): The DbtBouncerModelBase object to check.\n\n    Other Parameters:\n        exclude (Optional[str]): Regex pattern to match the model path. Model paths that match the pattern will not be checked.\n        include (Optional[str]): Regex pattern to match the model path. Only model paths that match the pattern will be checked.\n        severity (Optional[Literal[\"error\", \"warn\"]]): Severity level of the check. Default: `error`.\n\n    Example(s):\n        ```yaml\n        manifest_checks:\n            - name: check_model_documented_in_same_directory\n        ```\n\n    \"\"\"\n\n    model: \"DbtBouncerModelBase\" = Field(default=None)\n    name: Literal[\"check_model_documented_in_same_directory\"]\n\n    def execute(self) -> None:\n        \"\"\"Execute the check.\"\"\"\n        model_sql_dir = clean_path_str(self.model.original_file_path).split(\"/\")[:-1]\n        assert (  # noqa: PT018\n            hasattr(self.model, \"patch_path\")\n            and clean_path_str(self.model.patch_path) is not None\n        ), f\"`{self.model.name}` is not documented.\"\n\n        model_doc_dir = clean_path_str(\n            self.model.patch_path[\n                clean_path_str(self.model.patch_path).find(\"models\") :\n            ]\n        ).split(\"/\")[:-1]\n\n        assert (\n            model_doc_dir == model_sql_dir\n        ), f\"`{self.model.name}` is documented in a different directory to the `.sql` file: `{'/'.join(model_doc_dir)}` vs `{'/'.join(model_sql_dir)}`.\"\n
"},{"location":"checks/manifest/check_models/#manifest.check_models.CheckModelCodeDoesNotContainRegexpPattern","title":"CheckModelCodeDoesNotContainRegexpPattern","text":"

The raw code for a model must not match the specified regexp pattern.

Parameters:

Name Type Description Default regexp_pattern str

The regexp pattern that should not be matched by the model code.

required

Receives at execution time:

Name Type Description model DbtBouncerModelBase

The DbtBouncerModelBase object to check.

Other Parameters (passed via config file):

Name Type Description exclude Optional[str]

Regex pattern to match the model path. Model paths that match the pattern will not be checked.

include Optional[str]

Regex pattern to match the model path. Only model paths that match the pattern will be checked.

severity Optional[Literal['error', 'warn']]

Severity level of the check. Default: error.

Example(s):

manifest_checks:\n    # Prefer `coalesce` over `ifnull`: https://docs.sqlfluff.com/en/stable/rules.html#sqlfluff.rules.sphinx.Rule_CV02\n    - name: check_model_code_does_not_contain_regexp_pattern\n      regexp_pattern: .*[i][f][n][u][l][l].*\n

Source code in src/dbt_bouncer/checks/manifest/check_models.py
class CheckModelCodeDoesNotContainRegexpPattern(BaseCheck):\n    \"\"\"The raw code for a model must not match the specified regexp pattern.\n\n    Parameters:\n        regexp_pattern (str): The regexp pattern that should not be matched by the model code.\n\n    Receives:\n        model (DbtBouncerModelBase): The DbtBouncerModelBase object to check.\n\n    Other Parameters:\n        exclude (Optional[str]): Regex pattern to match the model path. Model paths that match the pattern will not be checked.\n        include (Optional[str]): Regex pattern to match the model path. Only model paths that match the pattern will be checked.\n        severity (Optional[Literal[\"error\", \"warn\"]]): Severity level of the check. Default: `error`.\n\n    Example(s):\n        ```yaml\n        manifest_checks:\n            # Prefer `coalesce` over `ifnull`: https://docs.sqlfluff.com/en/stable/rules.html#sqlfluff.rules.sphinx.Rule_CV02\n            - name: check_model_code_does_not_contain_regexp_pattern\n              regexp_pattern: .*[i][f][n][u][l][l].*\n        ```\n\n    \"\"\"\n\n    model: \"DbtBouncerModelBase\" = Field(default=None)\n    name: Literal[\"check_model_code_does_not_contain_regexp_pattern\"]\n    regexp_pattern: str\n\n    def execute(self) -> None:\n        \"\"\"Execute the check.\"\"\"\n        assert (\n            re.compile(self.regexp_pattern.strip(), flags=re.DOTALL).match(\n                self.model.raw_code\n            )\n            is None\n        ), f\"`{self.model.name}` contains a banned string: `{self.regexp_pattern.strip()}`.\"\n
"},{"location":"checks/manifest/check_models/#manifest.check_models.CheckModelDependsOnMultipleSources","title":"CheckModelDependsOnMultipleSources","text":"

Models cannot reference more than one source.

Parameters:

Name Type Description Default model DbtBouncerModelBase

The DbtBouncerModelBase object to check.

required

Other Parameters (passed via config file):

Name Type Description exclude Optional[str]

Regex pattern to match the model path. Model paths that match the pattern will not be checked.

include Optional[str]

Regex pattern to match the model path. Only model paths that match the pattern will be checked.

severity Optional[Literal['error', 'warn']]

Severity level of the check. Default: error.

Example(s):

manifest_checks:\n    - name: check_model_depends_on_multiple_sources\n

Source code in src/dbt_bouncer/checks/manifest/check_models.py
class CheckModelDependsOnMultipleSources(BaseCheck):\n    \"\"\"Models cannot reference more than one source.\n\n    Parameters:\n        model (DbtBouncerModelBase): The DbtBouncerModelBase object to check.\n\n    Other Parameters:\n        exclude (Optional[str]): Regex pattern to match the model path. Model paths that match the pattern will not be checked.\n        include (Optional[str]): Regex pattern to match the model path. Only model paths that match the pattern will be checked.\n        severity (Optional[Literal[\"error\", \"warn\"]]): Severity level of the check. Default: `error`.\n\n    Example(s):\n        ```yaml\n        manifest_checks:\n            - name: check_model_depends_on_multiple_sources\n        ```\n\n    \"\"\"\n\n    model: \"DbtBouncerModelBase\" = Field(default=None)\n    name: Literal[\"check_model_depends_on_multiple_sources\"]\n\n    def execute(self) -> None:\n        \"\"\"Execute the check.\"\"\"\n        num_reffed_sources = sum(\n            x.split(\".\")[0] == \"source\" for x in self.model.depends_on.nodes\n        )\n        assert (\n            num_reffed_sources <= 1\n        ), f\"`{self.model.name}` references more than one source.\"\n
"},{"location":"checks/manifest/check_models/#manifest.check_models.CheckModelDirectories","title":"CheckModelDirectories","text":"

Only specified sub-directories are permitted.

Parameters:

Name Type Description Default include str

Regex pattern to the directory to check.

required permitted_sub_directories List[str]

List of permitted sub-directories.

required

Receives at execution time:

Name Type Description model DbtBouncerModelBase

The DbtBouncerModelBase object to check.

Example(s):

manifest_checks:\n- name: check_model_directories\n  include: models\n  permitted_sub_directories:\n    - intermediate\n    - marts\n    - staging\n
# Restrict sub-directories within `./models/staging`\n- name: check_model_directories\n  include: ^models/staging\n  permitted_sub_directories:\n    - crm\n    - payments\n

Source code in src/dbt_bouncer/checks/manifest/check_models.py
class CheckModelDirectories(BaseCheck):\n    \"\"\"Only specified sub-directories are permitted.\n\n    Parameters:\n        include (str): Regex pattern to the directory to check.\n        permitted_sub_directories (List[str]): List of permitted sub-directories.\n\n    Receives:\n        model (DbtBouncerModelBase): The DbtBouncerModelBase object to check.\n\n    Example(s):\n        ```yaml\n        manifest_checks:\n        - name: check_model_directories\n          include: models\n          permitted_sub_directories:\n            - intermediate\n            - marts\n            - staging\n        ```\n        ```yaml\n        # Restrict sub-directories within `./models/staging`\n        - name: check_model_directories\n          include: ^models/staging\n          permitted_sub_directories:\n            - crm\n            - payments\n        ```\n\n    \"\"\"\n\n    include: str\n    model: \"DbtBouncerModelBase\" = Field(default=None)\n    name: Literal[\"check_model_directories\"]\n    permitted_sub_directories: List[str]\n\n    def execute(self) -> None:\n        \"\"\"Execute the check.\"\"\"\n        matched_path = re.compile(self.include.strip()).match(\n            clean_path_str(self.model.original_file_path)\n        )\n        path_after_match = clean_path_str(self.model.original_file_path)[\n            matched_path.end() + 1 :\n        ]\n\n        assert (\n            path_after_match.split(\"/\")[0] in self.permitted_sub_directories\n        ), f\"`{self.model.name}` is located in `{self.model.original_file_path.split('/')[1]}`, this is not a valid sub-directory.\"\n
"},{"location":"checks/manifest/check_models/#manifest.check_models.CheckModelHasContractsEnforced","title":"CheckModelHasContractsEnforced","text":"

Model must have contracts enforced.

Receives at execution time:

Name Type Description model DbtBouncerModelBase

The DbtBouncerModelBase object to check.

Other Parameters (passed via config file):

Name Type Description exclude Optional[str]

Regex pattern to match the model path. Model paths that match the pattern will not be checked.

include Optional[str]

Regex pattern to match the model path. Only model paths that match the pattern will be checked.

severity Optional[Literal['error', 'warn']]

Severity level of the check. Default: error.

Example(s):

manifest_checks:\n    - name: check_model_has_contracts_enforced\n      include: ^models/marts\n

Source code in src/dbt_bouncer/checks/manifest/check_models.py
class CheckModelHasContractsEnforced(BaseCheck):\n    \"\"\"Model must have contracts enforced.\n\n    Receives:\n        model (DbtBouncerModelBase): The DbtBouncerModelBase object to check.\n\n    Other Parameters:\n        exclude (Optional[str]): Regex pattern to match the model path. Model paths that match the pattern will not be checked.\n        include (Optional[str]): Regex pattern to match the model path. Only model paths that match the pattern will be checked.\n        severity (Optional[Literal[\"error\", \"warn\"]]): Severity level of the check. Default: `error`.\n\n    Example(s):\n        ```yaml\n        manifest_checks:\n            - name: check_model_has_contracts_enforced\n              include: ^models/marts\n        ```\n\n    \"\"\"\n\n    model: \"DbtBouncerModelBase\" = Field(default=None)\n    name: Literal[\"check_model_has_contracts_enforced\"]\n\n    def execute(self) -> None:\n        \"\"\"Execute the check.\"\"\"\n        assert (\n            self.model.contract.enforced is True\n        ), f\"`{self.model.name}` does not have contracts enforced.\"\n
"},{"location":"checks/manifest/check_models/#manifest.check_models.CheckModelHasMetaKeys","title":"CheckModelHasMetaKeys","text":"

The meta config for models must have the specified keys.

Parameters:

Name Type Description Default keys NestedDict

A list (that may contain sub-lists) of required keys.

required model DbtBouncerModelBase

The DbtBouncerModelBase object to check.

required

Other Parameters (passed via config file):

Name Type Description exclude Optional[str]

Regex pattern to match the model path. Model paths that match the pattern will not be checked.

include Optional[str]

Regex pattern to match the model path. Only model paths that match the pattern will be checked.

severity Optional[Literal['error', 'warn']]

Severity level of the check. Default: error.

Example(s):

manifest_checks:\n    - name: check_model_has_meta_keys\n      keys:\n        - maturity\n        - owner\n

Source code in src/dbt_bouncer/checks/manifest/check_models.py
class CheckModelHasMetaKeys(BaseCheck):\n    \"\"\"The `meta` config for models must have the specified keys.\n\n    Parameters:\n        keys (NestedDict): A list (that may contain sub-lists) of required keys.\n        model (DbtBouncerModelBase): The DbtBouncerModelBase object to check.\n\n    Other Parameters:\n        exclude (Optional[str]): Regex pattern to match the model path. Model paths that match the pattern will not be checked.\n        include (Optional[str]): Regex pattern to match the model path. Only model paths that match the pattern will be checked.\n        severity (Optional[Literal[\"error\", \"warn\"]]): Severity level of the check. Default: `error`.\n\n    Example(s):\n        ```yaml\n        manifest_checks:\n            - name: check_model_has_meta_keys\n              keys:\n                - maturity\n                - owner\n        ```\n\n    \"\"\"\n\n    keys: NestedDict\n    model: \"DbtBouncerModelBase\" = Field(default=None)\n    name: Literal[\"check_model_has_meta_keys\"]\n\n    def execute(self) -> None:\n        \"\"\"Execute the check.\"\"\"\n        missing_keys = find_missing_meta_keys(\n            meta_config=self.model.meta,\n            required_keys=self.keys.model_dump(),\n        )\n        assert (\n            missing_keys == []\n        ), f\"`{self.model.name}` is missing the following keys from the `meta` config: {[x.replace('>>', '') for x in missing_keys]}\"\n
"},{"location":"checks/manifest/check_models/#manifest.check_models.CheckModelHasNoUpstreamDependencies","title":"CheckModelHasNoUpstreamDependencies","text":"

Identify if models have no upstream dependencies as this likely indicates hard-coded tables references.

Receives at execution time:

Name Type Description model DbtBouncerModelBase

The DbtBouncerModelBase object to check.

Other Parameters (passed via config file):

Name Type Description exclude Optional[str]

Regex pattern to match the model path. Model paths that match the pattern will not be checked.

include Optional[str]

Regex pattern to match the model path. Only model paths that match the pattern will be checked.

severity Optional[Literal['error', 'warn']]

Severity level of the check. Default: error.

Example(s):

manifest_checks:\n    - name: check_model_has_no_upstream_dependencies\n

Source code in src/dbt_bouncer/checks/manifest/check_models.py
class CheckModelHasNoUpstreamDependencies(BaseCheck):\n    \"\"\"Identify if models have no upstream dependencies as this likely indicates hard-coded tables references.\n\n    Receives:\n        model (DbtBouncerModelBase): The DbtBouncerModelBase object to check.\n\n    Other Parameters:\n        exclude (Optional[str]): Regex pattern to match the model path. Model paths that match the pattern will not be checked.\n        include (Optional[str]): Regex pattern to match the model path. Only model paths that match the pattern will be checked.\n        severity (Optional[Literal[\"error\", \"warn\"]]): Severity level of the check. Default: `error`.\n\n    Example(s):\n        ```yaml\n        manifest_checks:\n            - name: check_model_has_no_upstream_dependencies\n        ```\n\n    \"\"\"\n\n    model: \"DbtBouncerModelBase\" = Field(default=None)\n    name: Literal[\"check_model_has_no_upstream_dependencies\"]\n\n    def execute(self) -> None:\n        \"\"\"Execute the check.\"\"\"\n        assert (\n            len(self.model.depends_on.nodes) > 0\n        ), f\"`{self.model.name}` has no upstream dependencies, this likely indicates hard-coded tables references.\"\n
"},{"location":"checks/manifest/check_models/#manifest.check_models.CheckModelHasTags","title":"CheckModelHasTags","text":"

Models must have the specified tags.

Parameters:

Name Type Description Default model DbtBouncerModelBase

The DbtBouncerModelBase object to check.

required tags List[str]

List of tags to check for.

required

Other Parameters (passed via config file):

Name Type Description exclude Optional[str]

Regex pattern to match the model path. Model paths that match the pattern will not be checked.

include Optional[str]

Regex pattern to match the model path. Only model paths that match the pattern will be checked.

severity Optional[Literal['error', 'warn']]

Severity level of the check. Default: error.

Example(s):

manifest_checks:\n    - name: check_model_has_tags\n      tags:\n        - tag_1\n        - tag_2\n

Source code in src/dbt_bouncer/checks/manifest/check_models.py
class CheckModelHasTags(BaseCheck):\n    \"\"\"Models must have the specified tags.\n\n    Parameters:\n        model (DbtBouncerModelBase): The DbtBouncerModelBase object to check.\n        tags (List[str]): List of tags to check for.\n\n    Other Parameters:\n        exclude (Optional[str]): Regex pattern to match the model path. Model paths that match the pattern will not be checked.\n        include (Optional[str]): Regex pattern to match the model path. Only model paths that match the pattern will be checked.\n        severity (Optional[Literal[\"error\", \"warn\"]]): Severity level of the check. Default: `error`.\n\n    Example(s):\n        ```yaml\n        manifest_checks:\n            - name: check_model_has_tags\n              tags:\n                - tag_1\n                - tag_2\n        ```\n\n    \"\"\"\n\n    model: \"DbtBouncerModelBase\" = Field(default=None)\n    name: Literal[\"check_model_has_tags\"]\n    tags: List[str]\n\n    def execute(self) -> None:\n        \"\"\"Execute the check.\"\"\"\n        missing_tags = [tag for tag in self.tags if tag not in self.model.tags]\n        assert (\n            not missing_tags\n        ), f\"`{self.model.name}` is missing required tags: {missing_tags}.\"\n
"},{"location":"checks/manifest/check_models/#manifest.check_models.CheckModelHasUniqueTest","title":"CheckModelHasUniqueTest","text":"

Models must have a test for uniqueness of a column.

Parameters:

Name Type Description Default accepted_uniqueness_tests Optional[List[str]]

List of tests that are accepted as uniqueness tests.

required model DbtBouncerModelBase

The DbtBouncerModelBase object to check.

required tests List[DbtBouncerTestBase]

List of DbtBouncerTestBase objects parsed from manifest.json.

required

Other Parameters (passed via config file):

Name Type Description exclude Optional[str]

Regex pattern to match the model path. Model paths that match the pattern will not be checked.

include Optional[str]

Regex pattern to match the model path. Only model paths that match the pattern will be checked.

severity Optional[Literal['error', 'warn']]

Severity level of the check. Default: error.

Example(s):

manifest_checks:\n    - name: check_model_has_unique_test\n      include: ^models/marts\n
manifest_checks:\n# Example of allowing a custom uniqueness test\n    - name: check_model_has_unique_test\n      accepted_uniqueness_tests:\n        - expect_compound_columns_to_be_unique\n        - my_custom_uniqueness_test\n        - unique\n

Source code in src/dbt_bouncer/checks/manifest/check_models.py
class CheckModelHasUniqueTest(BaseCheck):\n    \"\"\"Models must have a test for uniqueness of a column.\n\n    Parameters:\n        accepted_uniqueness_tests (Optional[List[str]]): List of tests that are accepted as uniqueness tests.\n        model (DbtBouncerModelBase): The DbtBouncerModelBase object to check.\n        tests (List[DbtBouncerTestBase]): List of DbtBouncerTestBase objects parsed from `manifest.json`.\n\n    Other Parameters:\n        exclude (Optional[str]): Regex pattern to match the model path. Model paths that match the pattern will not be checked.\n        include (Optional[str]): Regex pattern to match the model path. Only model paths that match the pattern will be checked.\n        severity (Optional[Literal[\"error\", \"warn\"]]): Severity level of the check. Default: `error`.\n\n    Example(s):\n        ```yaml\n        manifest_checks:\n            - name: check_model_has_unique_test\n              include: ^models/marts\n        ```\n        ```yaml\n        manifest_checks:\n        # Example of allowing a custom uniqueness test\n            - name: check_model_has_unique_test\n              accepted_uniqueness_tests:\n                - expect_compound_columns_to_be_unique\n                - my_custom_uniqueness_test\n                - unique\n        ```\n\n    \"\"\"\n\n    accepted_uniqueness_tests: Optional[List[str]] = Field(\n        default=[\n            \"expect_compound_columns_to_be_unique\",\n            \"dbt_utils.unique_combination_of_columns\",\n            \"unique\",\n        ],\n    )\n    model: \"DbtBouncerModelBase\" = Field(default=None)\n    name: Literal[\"check_model_has_unique_test\"]\n    tests: List[\"DbtBouncerTestBase\"] = Field(default=[])\n\n    def execute(self) -> None:\n        \"\"\"Execute the check.\"\"\"\n        num_unique_tests = sum(\n            test.attached_node == self.model.unique_id\n            and test.test_metadata.name in self.accepted_uniqueness_tests  # type: ignore[operator]\n            for test in self.tests\n            if hasattr(test, \"test_metadata\")\n        )\n        assert (\n            num_unique_tests >= 1\n        ), f\"`{self.model.name}` does not have a test for uniqueness of a column.\"\n
"},{"location":"checks/manifest/check_models/#manifest.check_models.CheckModelHasUnitTests","title":"CheckModelHasUnitTests","text":"

Models must have more than the specified number of unit tests.

Parameters:

Name Type Description Default min_number_of_unit_tests Optional[int]

The minimum number of unit tests that a model must have.

required

Receives at execution time:

Name Type Description manifest_obj DbtBouncerManifest

The DbtBouncerManifest object parsed from manifest.json.

model DbtBouncerModelBase

The DbtBouncerModelBase object to check.

unit_tests List[UnitTests]

List of UnitTests objects parsed from manifest.json.

Other Parameters (passed via config file):

Name Type Description exclude Optional[str]

Regex pattern to match the model path. Model paths that match the pattern will not be checked.

include Optional[str]

Regex pattern to match the model path. Only model paths that match the pattern will be checked.

severity Optional[Literal['error', 'warn']]

Severity level of the check. Default: error.

Warning

This check is only supported for dbt 1.8.0 and above.

Example(s):

manifest_checks:\n    - name: check_model_has_unit_tests\n      include: ^models/marts\n
manifest_checks:\n    - name: check_model_has_unit_tests\n      min_number_of_unit_tests: 2\n

Source code in src/dbt_bouncer/checks/manifest/check_models.py
class CheckModelHasUnitTests(BaseCheck):\n    \"\"\"Models must have more than the specified number of unit tests.\n\n    Parameters:\n        min_number_of_unit_tests (Optional[int]): The minimum number of unit tests that a model must have.\n\n    Receives:\n        manifest_obj (DbtBouncerManifest): The DbtBouncerManifest object parsed from `manifest.json`.\n        model (DbtBouncerModelBase): The DbtBouncerModelBase object to check.\n        unit_tests (List[UnitTests]): List of UnitTests objects parsed from `manifest.json`.\n\n    Other Parameters:\n        exclude (Optional[str]): Regex pattern to match the model path. Model paths that match the pattern will not be checked.\n        include (Optional[str]): Regex pattern to match the model path. Only model paths that match the pattern will be checked.\n        severity (Optional[Literal[\"error\", \"warn\"]]): Severity level of the check. Default: `error`.\n\n    !!! warning\n\n        This check is only supported for dbt 1.8.0 and above.\n\n    Example(s):\n        ```yaml\n        manifest_checks:\n            - name: check_model_has_unit_tests\n              include: ^models/marts\n        ```\n        ```yaml\n        manifest_checks:\n            - name: check_model_has_unit_tests\n              min_number_of_unit_tests: 2\n        ```\n\n    \"\"\"\n\n    manifest_obj: \"DbtBouncerManifest\" = Field(default=None)\n    min_number_of_unit_tests: int = Field(default=1)\n    model: \"DbtBouncerModelBase\" = Field(default=None)\n    name: Literal[\"check_model_has_unit_tests\"]\n    unit_tests: List[\"UnitTests\"] = Field(default=[])\n\n    def execute(self) -> None:\n        \"\"\"Execute the check.\"\"\"\n        if (\n            semver.Version.parse(self.manifest_obj.manifest.metadata.dbt_version)\n            >= \"1.8.0\"\n        ):\n            num_unit_tests = len(\n                [\n                    t.unique_id\n                    for t in self.unit_tests\n                    if t.depends_on.nodes[0] == self.model.unique_id\n                ],\n            )\n            assert (\n                num_unit_tests >= self.min_number_of_unit_tests\n            ), f\"`{self.model.name}` has {num_unit_tests} unit tests, this is less than the minimum of {self.min_number_of_unit_tests}.\"\n        else:\n            logging.warning(\n                \"The `check_model_has_unit_tests` check is only supported for dbt 1.8.0 and above.\",\n            )\n
"},{"location":"checks/manifest/check_models/#manifest.check_models.CheckModelMaxChainedViews","title":"CheckModelMaxChainedViews","text":"

Models cannot have more than the specified number of upstream dependents that are not tables.

Parameters:

Name Type Description Default materializations_to_include Optional[List[str]]

List of materializations to include in the check.

required max_chained_views Optional[int]

The maximum number of upstream dependents that are not tables.

required

Receives at execution time:

Name Type Description model DbtBouncerModelBase

The DbtBouncerModelBase object to check.

models List[DbtBouncerModelBase]

List of DbtBouncerModelBase objects parsed from manifest.json.

Other Parameters (passed via config file):

Name Type Description exclude Optional[str]

Regex pattern to match the model path. Model paths that match the pattern will not be checked.

include Optional[str]

Regex pattern to match the model path. Only model paths that match the pattern will be checked.

severity Optional[Literal['error', 'warn']]

Severity level of the check. Default: error.

Example(s):

manifest_checks:\n    - name: check_model_max_chained_views\n
manifest_checks:\n    - name: check_model_max_chained_views\n      materializations_to_include:\n        - ephemeral\n        - my_custom_materialization\n        - view\n      max_chained_views: 5\n

Source code in src/dbt_bouncer/checks/manifest/check_models.py
class CheckModelMaxChainedViews(BaseCheck):\n    \"\"\"Models cannot have more than the specified number of upstream dependents that are not tables.\n\n    Parameters:\n        materializations_to_include (Optional[List[str]]): List of materializations to include in the check.\n        max_chained_views (Optional[int]): The maximum number of upstream dependents that are not tables.\n\n    Receives:\n        model (DbtBouncerModelBase): The DbtBouncerModelBase object to check.\n        models (List[DbtBouncerModelBase]): List of DbtBouncerModelBase objects parsed from `manifest.json`.\n\n    Other Parameters:\n        exclude (Optional[str]): Regex pattern to match the model path. Model paths that match the pattern will not be checked.\n        include (Optional[str]): Regex pattern to match the model path. Only model paths that match the pattern will be checked.\n        severity (Optional[Literal[\"error\", \"warn\"]]): Severity level of the check. Default: `error`.\n\n    Example(s):\n        ```yaml\n        manifest_checks:\n            - name: check_model_max_chained_views\n        ```\n        ```yaml\n        manifest_checks:\n            - name: check_model_max_chained_views\n              materializations_to_include:\n                - ephemeral\n                - my_custom_materialization\n                - view\n              max_chained_views: 5\n        ```\n\n    \"\"\"\n\n    manifest_obj: \"DbtBouncerManifest\" = Field(default=None)\n    materializations_to_include: List[str] = Field(\n        default=[\"ephemeral\", \"view\"],\n    )\n    max_chained_views: int = Field(\n        default=3,\n    )\n    model: \"DbtBouncerModelBase\" = Field(default=None)\n    models: List[\"DbtBouncerModelBase\"] = Field(default=[])\n    name: Literal[\"check_model_max_chained_views\"]\n\n    def execute(self) -> None:\n        \"\"\"Execute the check.\"\"\"\n\n        def return_upstream_view_models(\n            materializations,\n            max_chained_views,\n            models,\n            model_unique_ids_to_check,\n            package_name,\n            depth=0,\n        ):\n            \"\"\"Recursive function to return model unique_id's of upstream models that are views. Depth of recursion can be specified. If no models meet the criteria then an empty list is returned.\n\n            Returns\n            -\n                List[str]: List of model unique_id's of upstream models that are views.\n\n            \"\"\"\n            if depth == max_chained_views or model_unique_ids_to_check == []:\n                return model_unique_ids_to_check\n\n            relevant_upstream_models = []\n            for model in model_unique_ids_to_check:\n                upstream_nodes = list(\n                    next(m2 for m2 in models if m2.unique_id == model).depends_on.nodes,\n                )\n                if upstream_nodes != []:\n                    upstream_models = [\n                        m\n                        for m in upstream_nodes\n                        if m.split(\".\")[0] == \"model\"\n                        and m.split(\".\")[1] == package_name\n                    ]\n                    for i in upstream_models:\n                        if (\n                            next(\n                                m for m in models if m.unique_id == i\n                            ).config.materialized\n                            in materializations\n                        ):\n                            relevant_upstream_models.append(i)\n\n            depth += 1\n            return return_upstream_view_models(\n                materializations=materializations,\n                max_chained_views=max_chained_views,\n                models=models,\n                model_unique_ids_to_check=relevant_upstream_models,\n                package_name=package_name,\n                depth=depth,\n            )\n\n        assert (\n            len(\n                return_upstream_view_models(\n                    materializations=self.materializations_to_include,\n                    max_chained_views=self.max_chained_views,\n                    models=self.models,\n                    model_unique_ids_to_check=[self.model.unique_id],\n                    package_name=self.manifest_obj.manifest.metadata.project_name,\n                ),\n            )\n            == 0\n        ), f\"`{self.model.name}` has more than {self.max_chained_views} upstream dependents that are not tables.\"\n
"},{"location":"checks/manifest/check_models/#manifest.check_models.CheckModelMaxFanout","title":"CheckModelMaxFanout","text":"

Models cannot have more than the specified number of downstream models.

Parameters:

Name Type Description Default max_downstream_models Optional[int]

The maximum number of permitted downstream models.

required

Receives at execution time:

Name Type Description model DbtBouncerModelBase

The DbtBouncerModelBase object to check.

models List[DbtBouncerModelBase]

List of DbtBouncerModelBase objects parsed from manifest.json.

Other Parameters (passed via config file):

Name Type Description exclude Optional[str]

Regex pattern to match the model path. Model paths that match the pattern will not be checked.

include Optional[str]

Regex pattern to match the model path. Only model paths that match the pattern will be checked.

severity Optional[Literal['error', 'warn']]

Severity level of the check. Default: error.

Example(s):

manifest_checks:\n    - name: check_model_max_fanout\n      max_downstream_models: 2\n

Source code in src/dbt_bouncer/checks/manifest/check_models.py
class CheckModelMaxFanout(BaseCheck):\n    \"\"\"Models cannot have more than the specified number of downstream models.\n\n    Parameters:\n        max_downstream_models (Optional[int]): The maximum number of permitted downstream models.\n\n    Receives:\n        model (DbtBouncerModelBase): The DbtBouncerModelBase object to check.\n        models (List[DbtBouncerModelBase]): List of DbtBouncerModelBase objects parsed from `manifest.json`.\n\n    Other Parameters:\n        exclude (Optional[str]): Regex pattern to match the model path. Model paths that match the pattern will not be checked.\n        include (Optional[str]): Regex pattern to match the model path. Only model paths that match the pattern will be checked.\n        severity (Optional[Literal[\"error\", \"warn\"]]): Severity level of the check. Default: `error`.\n\n    Example(s):\n        ```yaml\n        manifest_checks:\n            - name: check_model_max_fanout\n              max_downstream_models: 2\n        ```\n\n    \"\"\"\n\n    max_downstream_models: int = Field(default=3)\n    model: \"DbtBouncerModelBase\" = Field(default=None)\n    models: List[\"DbtBouncerModelBase\"] = Field(default=[])\n    name: Literal[\"check_model_max_fanout\"]\n\n    def execute(self) -> None:\n        \"\"\"Execute the check.\"\"\"\n        num_downstream_models = sum(\n            self.model.unique_id in m.depends_on.nodes for m in self.models\n        )\n\n        assert (\n            num_downstream_models <= self.max_downstream_models\n        ), f\"`{self.model.name}` has {num_downstream_models} downstream models, which is more than the permitted maximum of {self.max_downstream_models}.\"\n
"},{"location":"checks/manifest/check_models/#manifest.check_models.CheckModelMaxNumberOfLines","title":"CheckModelMaxNumberOfLines","text":"

Models may not have more than the specified number of lines.

Parameters:

Name Type Description Default max_number_of_lines int

The maximum number of permitted lines.

required model DbtBouncerModelBase

The DbtBouncerModelBase object to check.

required

Other Parameters (passed via config file):

Name Type Description exclude Optional[str]

Regex pattern to match the model path. Model paths that match the pattern will not be checked.

include Optional[str]

Regex pattern to match the model path. Only model paths that match the pattern will be checked.

severity Optional[Literal['error', 'warn']]

Severity level of the check. Default: error.

Example(s):

manifest_checks:\n    - name: check_model_max_number_of_lines\n
manifest_checks:\n    - name: check_model_max_number_of_lines\n      max_number_of_lines: 150\n

Source code in src/dbt_bouncer/checks/manifest/check_models.py
class CheckModelMaxNumberOfLines(BaseCheck):\n    \"\"\"Models may not have more than the specified number of lines.\n\n    Parameters:\n        max_number_of_lines (int): The maximum number of permitted lines.\n\n        model (DbtBouncerModelBase): The DbtBouncerModelBase object to check.\n\n    Other Parameters:\n        exclude (Optional[str]): Regex pattern to match the model path. Model paths that match the pattern will not be checked.\n        include (Optional[str]): Regex pattern to match the model path. Only model paths that match the pattern will be checked.\n        severity (Optional[Literal[\"error\", \"warn\"]]): Severity level of the check. Default: `error`.\n\n    Example(s):\n        ```yaml\n        manifest_checks:\n            - name: check_model_max_number_of_lines\n        ```\n        ```yaml\n        manifest_checks:\n            - name: check_model_max_number_of_lines\n              max_number_of_lines: 150\n        ```\n\n    \"\"\"\n\n    model: \"DbtBouncerModelBase\" = Field(default=None)\n    name: Literal[\"check_model_max_number_of_lines\"]\n    max_number_of_lines: int = Field(default=100)\n\n    def execute(self) -> None:\n        \"\"\"Execute the check.\"\"\"\n        actual_number_of_lines = self.model.raw_code.count(\"\\n\") + 1\n\n        assert (\n            actual_number_of_lines <= self.max_number_of_lines\n        ), f\"`{self.model.name}` has {actual_number_of_lines} lines, this is more than the maximum permitted number of lines ({self.max_number_of_lines}).\"\n
"},{"location":"checks/manifest/check_models/#manifest.check_models.CheckModelMaxUpstreamDependencies","title":"CheckModelMaxUpstreamDependencies","text":"

Limit the number of upstream dependencies a model has.

Parameters:

Name Type Description Default max_upstream_macros Optional[int]

The maximum number of permitted upstream macros.

required max_upstream_models Optional[int]

The maximum number of permitted upstream models.

required max_upstream_sources Optional[int]

The maximum number of permitted upstream sources.

required

Receives at execution time:

Name Type Description model DbtBouncerModelBase

The DbtBouncerModelBase object to check.

Other Parameters (passed via config file):

Name Type Description exclude Optional[str]

Regex pattern to match the model path. Model paths that match the pattern will not be checked.

include Optional[str]

Regex pattern to match the model path. Only model paths that match the pattern will be checked.

severity Optional[Literal['error', 'warn']]

Severity level of the check. Default: error.

Example(s):

manifest_checks:\n    - name: check_model_max_upstream_dependencies\n      max_upstream_models: 3\n

Source code in src/dbt_bouncer/checks/manifest/check_models.py
class CheckModelMaxUpstreamDependencies(BaseCheck):\n    \"\"\"Limit the number of upstream dependencies a model has.\n\n    Parameters:\n        max_upstream_macros (Optional[int]): The maximum number of permitted upstream macros.\n        max_upstream_models (Optional[int]): The maximum number of permitted upstream models.\n        max_upstream_sources (Optional[int]): The maximum number of permitted upstream sources.\n\n    Receives:\n        model (DbtBouncerModelBase): The DbtBouncerModelBase object to check.\n\n    Other Parameters:\n        exclude (Optional[str]): Regex pattern to match the model path. Model paths that match the pattern will not be checked.\n        include (Optional[str]): Regex pattern to match the model path. Only model paths that match the pattern will be checked.\n        severity (Optional[Literal[\"error\", \"warn\"]]): Severity level of the check. Default: `error`.\n\n    Example(s):\n        ```yaml\n        manifest_checks:\n            - name: check_model_max_upstream_dependencies\n              max_upstream_models: 3\n        ```\n\n    \"\"\"\n\n    max_upstream_macros: int = Field(\n        default=5,\n    )\n    max_upstream_models: int = Field(\n        default=5,\n    )\n    max_upstream_sources: int = Field(\n        default=1,\n    )\n    model: \"DbtBouncerModelBase\" = Field(default=None)\n    name: Literal[\"check_model_max_upstream_dependencies\"]\n\n    def execute(self) -> None:\n        \"\"\"Execute the check.\"\"\"\n        num_upstream_macros = len(list(self.model.depends_on.macros))\n        num_upstream_models = len(\n            [m for m in self.model.depends_on.nodes if m.split(\".\")[0] == \"model\"],\n        )\n        num_upstream_sources = len(\n            [m for m in self.model.depends_on.nodes if m.split(\".\")[0] == \"source\"],\n        )\n\n        assert (\n            num_upstream_macros <= self.max_upstream_macros\n        ), f\"`{self.model.name}` has {num_upstream_macros} upstream macros, which is more than the permitted maximum of {self.max_upstream_macros}.\"\n        assert (\n            num_upstream_models <= self.max_upstream_models\n        ), f\"`{self.model.name}` has {num_upstream_models} upstream models, which is more than the permitted maximum of {self.max_upstream_models}.\"\n        assert (\n            num_upstream_sources <= self.max_upstream_sources\n        ), f\"`{self.model.name}` has {num_upstream_sources} upstream sources, which is more than the permitted maximum of {self.max_upstream_sources}.\"\n
"},{"location":"checks/manifest/check_models/#manifest.check_models.CheckModelNames","title":"CheckModelNames","text":"

Models must have a name that matches the supplied regex.

Parameters:

Name Type Description Default model_name_pattern str

Regexp the model name must match.

required

Receives at execution time:

Name Type Description model DbtBouncerModelBase

The DbtBouncerModelBase object to check.

Other Parameters (passed via config file):

Name Type Description exclude Optional[str]

Regex pattern to match the model path. Model paths that match the pattern will not be checked.

include Optional[str]

Regex pattern to match the model path. Only model paths that match the pattern will be checked.

severity Optional[Literal['error', 'warn']]

Severity level of the check. Default: error.

Example(s):

manifest_checks:\n    - name: check_model_names\n      include: ^models/intermediate\n      model_name_pattern: ^int_\n    - name: check_model_names\n      include: ^models/staging\n      model_name_pattern: ^stg_\n

Source code in src/dbt_bouncer/checks/manifest/check_models.py
class CheckModelNames(BaseCheck):\n    \"\"\"Models must have a name that matches the supplied regex.\n\n    Parameters:\n        model_name_pattern (str): Regexp the model name must match.\n\n    Receives:\n        model (DbtBouncerModelBase): The DbtBouncerModelBase object to check.\n\n    Other Parameters:\n        exclude (Optional[str]): Regex pattern to match the model path. Model paths that match the pattern will not be checked.\n        include (Optional[str]): Regex pattern to match the model path. Only model paths that match the pattern will be checked.\n        severity (Optional[Literal[\"error\", \"warn\"]]): Severity level of the check. Default: `error`.\n\n    Example(s):\n        ```yaml\n        manifest_checks:\n            - name: check_model_names\n              include: ^models/intermediate\n              model_name_pattern: ^int_\n            - name: check_model_names\n              include: ^models/staging\n              model_name_pattern: ^stg_\n        ```\n\n    \"\"\"\n\n    model_config = ConfigDict(extra=\"forbid\", protected_namespaces=())\n\n    model: \"DbtBouncerModelBase\" = Field(default=None)\n    name: Literal[\"check_model_names\"]\n    model_name_pattern: str\n\n    def execute(self) -> None:\n        \"\"\"Execute the check.\"\"\"\n        assert (\n            re.compile(self.model_name_pattern.strip()).match(self.model.name)\n            is not None\n        ), f\"`{self.model.name}` does not match the supplied regex `{self.model_name_pattern.strip()})`.\"\n
"},{"location":"checks/manifest/check_models/#manifest.check_models.CheckModelPropertyFileLocation","title":"CheckModelPropertyFileLocation","text":"

Model properties files must follow the guidance provided by dbt here.

Parameters:

Name Type Description Default model DbtBouncerModelBase

The DbtBouncerModelBase object to check.

required

Other Parameters (passed via config file):

Name Type Description exclude Optional[str]

Regex pattern to match the model path. Model paths that match the pattern will not be checked.

include Optional[str]

Regex pattern to match the model path. Only model paths that match the pattern will be checked.

severity Optional[Literal['error', 'warn']]

Severity level of the check. Default: error.

Example(s):

manifest_checks:\n    - name: check_model_property_file_location\n

Source code in src/dbt_bouncer/checks/manifest/check_models.py
class CheckModelPropertyFileLocation(BaseCheck):\n    \"\"\"Model properties files must follow the guidance provided by dbt [here](https://docs.getdbt.com/best-practices/how-we-structure/1-guide-overview).\n\n    Parameters:\n        model (DbtBouncerModelBase): The DbtBouncerModelBase object to check.\n\n    Other Parameters:\n        exclude (Optional[str]): Regex pattern to match the model path. Model paths that match the pattern will not be checked.\n        include (Optional[str]): Regex pattern to match the model path. Only model paths that match the pattern will be checked.\n        severity (Optional[Literal[\"error\", \"warn\"]]): Severity level of the check. Default: `error`.\n\n    Example(s):\n        ```yaml\n        manifest_checks:\n            - name: check_model_property_file_location\n        ```\n\n    \"\"\"\n\n    model: \"DbtBouncerModelBase\" = Field(default=None)\n    name: Literal[\"check_model_property_file_location\"]\n\n    def execute(self) -> None:\n        \"\"\"Execute the check.\"\"\"\n        assert (  # noqa: PT018\n            hasattr(self.model, \"patch_path\")\n            and clean_path_str(self.model.patch_path) is not None\n        ), f\"`{self.model.name}` is not documented.\"\n\n        expected_substr = (\n            \"_\".join(clean_path_str(self.model.original_file_path).split(\"/\")[1:-1])\n            .replace(\"staging\", \"stg\")\n            .replace(\"intermediate\", \"int\")\n            .replace(\"marts\", \"\")\n        )\n        properties_yml_name = clean_path_str(self.model.patch_path).split(\"/\")[-1]\n\n        assert properties_yml_name.startswith(\n            \"_\",\n        ), f\"The properties file for `{self.model.name}` (`{properties_yml_name}`) does not start with an underscore.\"\n        assert (\n            expected_substr in properties_yml_name\n        ), f\"The properties file for `{self.model.name}` (`{properties_yml_name}`) does not contain the expected substring (`{expected_substr}`).\"\n        assert properties_yml_name.endswith(\n            \"__models.yml\",\n        ), f\"The properties file for `{self.model.name}` (`{properties_yml_name}`) does not end with `__models.yml`.\"\n
"},{"location":"checks/manifest/check_models/#manifest.check_models.CheckModelsTestCoverage","title":"CheckModelsTestCoverage","text":"

Set the minimum percentage of models that have at least one test.

Parameters:

Name Type Description Default min_model_test_coverage_pct float

The minimum percentage of models that must have at least one test.

required models List[DbtBouncerModelBase]

List of DbtBouncerModelBase objects parsed from manifest.json.

required tests List[DbtBouncerTestBase]

List of DbtBouncerTestBase objects parsed from manifest.json.

required

Other Parameters (passed via config file):

Name Type Description severity Optional[Literal['error', 'warn']]

Severity level of the check. Default: error.

Example(s):

manifest_checks:\n    - name: check_model_test_coverage\n      min_model_test_coverage_pct: 90\n

Source code in src/dbt_bouncer/checks/manifest/check_models.py
class CheckModelsTestCoverage(BaseModel):\n    \"\"\"Set the minimum percentage of models that have at least one test.\n\n    Parameters:\n        min_model_test_coverage_pct (float): The minimum percentage of models that must have at least one test.\n        models (List[DbtBouncerModelBase]): List of DbtBouncerModelBase objects parsed from `manifest.json`.\n        tests (List[DbtBouncerTestBase]): List of DbtBouncerTestBase objects parsed from `manifest.json`.\n\n    Other Parameters:\n        severity (Optional[Literal[\"error\", \"warn\"]]): Severity level of the check. Default: `error`.\n\n\n    Example(s):\n        ```yaml\n        manifest_checks:\n            - name: check_model_test_coverage\n              min_model_test_coverage_pct: 90\n        ```\n\n    \"\"\"\n\n    model_config = ConfigDict(extra=\"forbid\")\n\n    index: Optional[int] = Field(\n        default=None,\n        description=\"Index to uniquely identify the check, calculated at runtime.\",\n    )\n    name: Literal[\"check_model_test_coverage\"]\n    min_model_test_coverage_pct: float = Field(\n        default=100,\n        ge=0,\n        le=100,\n    )\n    models: List[\"DbtBouncerModelBase\"] = Field(default=[])\n    severity: Optional[Literal[\"error\", \"warn\"]] = Field(\n        default=\"error\",\n        description=\"Severity of the check, one of 'error' or 'warn'.\",\n    )\n    tests: List[\"DbtBouncerTestBase\"] = Field(default=[])\n\n    def execute(self) -> None:\n        \"\"\"Execute the check.\"\"\"\n        num_models = len(self.models)\n        models_with_tests = []\n        for model in self.models:\n            for test in self.tests:\n                if model.unique_id in test.depends_on.nodes:\n                    models_with_tests.append(model.unique_id)\n        num_models_with_tests = len(set(models_with_tests))\n        model_test_coverage_pct = (num_models_with_tests / num_models) * 100\n\n        assert (\n            model_test_coverage_pct >= self.min_model_test_coverage_pct\n        ), f\"Only {model_test_coverage_pct}% of models have at least one test, this is less than the permitted minimum of {self.min_model_test_coverage_pct}%.\"\n
"},{"location":"checks/manifest/check_semantic_models/","title":"Manifest Checks: Semantic Models","text":"

Note

The below checks require manifest.json to be present.

"},{"location":"checks/manifest/check_semantic_models/#manifest.check_semantic_models.CheckSemanticModelOnNonPublicModels","title":"CheckSemanticModelOnNonPublicModels","text":"

Semantic models should be based on public models only.

Receives at execution time:

Name Type Description models List[DbtBouncerModelBase]

List of DbtBouncerModelBase objects parsed from manifest.json.

semantic_model DbtBouncerSemanticModelBase

The DbtBouncerSemanticModelBase object to check.

Other Parameters (passed via config file):

Name Type Description exclude Optional[str]

Regex pattern to match the semantic model path (i.e the .yml file where the semantic model is configured). Semantic model paths that match the pattern will not be checked.

include Optional[str]

Regex pattern to match the semantic model path (i.e the .yml file where the semantic model is configured). Only semantic model paths that match the pattern will be checked.

severity Optional[Literal['error', 'warn']]

Severity level of the check. Default: error.

Example(s):

manifest_checks:\n    - name: check_semantic_model_based_on_non_public_models\n

Source code in src/dbt_bouncer/checks/manifest/check_semantic_models.py
class CheckSemanticModelOnNonPublicModels(BaseCheck):\n    \"\"\"Semantic models should be based on public models only.\n\n    Receives:\n        models (List[DbtBouncerModelBase]): List of DbtBouncerModelBase objects parsed from `manifest.json`.\n        semantic_model (DbtBouncerSemanticModelBase): The DbtBouncerSemanticModelBase object to check.\n\n    Other Parameters:\n        exclude (Optional[str]): Regex pattern to match the semantic model path (i.e the .yml file where the semantic model is configured). Semantic model paths that match the pattern will not be checked.\n        include (Optional[str]): Regex pattern to match the semantic model path (i.e the .yml file where the semantic model is configured). Only semantic model paths that match the pattern will be checked.\n        severity (Optional[Literal[\"error\", \"warn\"]]): Severity level of the check. Default: `error`.\n\n    Example(s):\n        ```yaml\n        manifest_checks:\n            - name: check_semantic_model_based_on_non_public_models\n        ```\n\n    \"\"\"\n\n    models: List[\"DbtBouncerModelBase\"] = Field(default=[])\n    name: Literal[\"check_semantic_model_based_on_non_public_models\"]\n    semantic_model: \"DbtBouncerSemanticModelBase\" = Field(default=None)\n\n    def execute(self) -> None:\n        \"\"\"Execute the check.\"\"\"\n        non_public_upstream_dependencies = []\n        for model in self.semantic_model.depends_on.nodes:\n            if (\n                model.split(\".\")[0] == \"model\"\n                and model.split(\".\")[1] == self.semantic_model.package_name\n            ):\n                model = next(m for m in self.models if m.unique_id == model)\n                if model.access.value != \"public\":\n                    non_public_upstream_dependencies.append(model.name)\n\n        assert not non_public_upstream_dependencies, f\"Semantic model `{self.semantic_model.name}` is based on a model(s) that is not public: {non_public_upstream_dependencies}.\"\n
"},{"location":"checks/manifest/check_sources/","title":"Manifest Checks: Sources","text":"

Note

The below checks require manifest.json to be present.

"},{"location":"checks/manifest/check_sources/#manifest.check_sources.CheckSourceDescriptionPopulated","title":"CheckSourceDescriptionPopulated","text":"

Sources must have a populated description.

Receives at execution time:

Name Type Description source DbtBouncerSourceBase

The DbtBouncerSourceBase object to check.

Other Parameters (passed via config file):

Name Type Description exclude Optional[str]

Regex pattern to match the source path (i.e the .yml file where the source is configured). Source paths that match the pattern will not be checked.

include Optional[str]

Regex pattern to match the source path (i.e the .yml file where the source is configured). Only source paths that match the pattern will be checked.

severity Optional[Literal['error', 'warn']]

Severity level of the check. Default: error.

Example(s):

manifest_checks:\n    - name: check_source_description_populated\n

Source code in src/dbt_bouncer/checks/manifest/check_sources.py
class CheckSourceDescriptionPopulated(BaseCheck):\n    \"\"\"Sources must have a populated description.\n\n    Receives:\n        source (DbtBouncerSourceBase): The DbtBouncerSourceBase object to check.\n\n    Other Parameters:\n        exclude (Optional[str]): Regex pattern to match the source path (i.e the .yml file where the source is configured). Source paths that match the pattern will not be checked.\n        include (Optional[str]): Regex pattern to match the source path (i.e the .yml file where the source is configured). Only source paths that match the pattern will be checked.\n        severity (Optional[Literal[\"error\", \"warn\"]]): Severity level of the check. Default: `error`.\n\n    Example(s):\n        ```yaml\n        manifest_checks:\n            - name: check_source_description_populated\n        ```\n\n    \"\"\"\n\n    name: Literal[\"check_source_description_populated\"]\n    source: \"DbtBouncerSourceBase\" = Field(default=None)\n\n    def execute(self) -> None:\n        \"\"\"Execute the check.\"\"\"\n        assert (\n            len(self.source.description.strip()) > 4\n        ), f\"`{self.source.source_name}.{self.source.name}` does not have a populated description.\"\n
"},{"location":"checks/manifest/check_sources/#manifest.check_sources.CheckSourceFreshnessPopulated","title":"CheckSourceFreshnessPopulated","text":"

Sources must have a populated freshness.

Receives at execution time:

Name Type Description source DbtBouncerSource

The DbtBouncerSourceBase object to check.

Other Parameters (passed via config file):

Name Type Description exclude Optional[str]

Regex pattern to match the source path (i.e the .yml file where the source is configured). Source paths that match the pattern will not be checked.

include Optional[str]

Regex pattern to match the source path (i.e the .yml file where the source is configured). Only source paths that match the pattern will be checked.

severity Optional[Literal['error', 'warn']]

Severity level of the check. Default: error.

Example(s):

manifest_checks:\n    - name: check_source_freshness_populated\n

Source code in src/dbt_bouncer/checks/manifest/check_sources.py
class CheckSourceFreshnessPopulated(BaseCheck):\n    \"\"\"Sources must have a populated freshness.\n\n    Receives:\n        source (DbtBouncerSource): The DbtBouncerSourceBase object to check.\n\n    Other Parameters:\n        exclude (Optional[str]): Regex pattern to match the source path (i.e the .yml file where the source is configured). Source paths that match the pattern will not be checked.\n        include (Optional[str]): Regex pattern to match the source path (i.e the .yml file where the source is configured). Only source paths that match the pattern will be checked.\n        severity (Optional[Literal[\"error\", \"warn\"]]): Severity level of the check. Default: `error`.\n\n    Example(s):\n        ```yaml\n        manifest_checks:\n            - name: check_source_freshness_populated\n        ```\n\n    \"\"\"\n\n    name: Literal[\"check_source_freshness_populated\"]\n    source: \"DbtBouncerSourceBase\" = Field(default=None)\n\n    def execute(self) -> None:\n        \"\"\"Execute the check.\"\"\"\n        error_msg = f\"`{self.source.source_name}.{self.source.name}` does not have a populated freshness.\"\n        assert self.source.freshness is not None, error_msg\n        assert (\n            self.source.freshness.error_after.count is not None\n            and self.source.freshness.error_after.period is not None\n        ) or (\n            self.source.freshness.warn_after.count is not None\n            and self.source.freshness.warn_after.period is not None\n        ), error_msg\n
"},{"location":"checks/manifest/check_sources/#manifest.check_sources.CheckSourceHasMetaKeys","title":"CheckSourceHasMetaKeys","text":"

The meta config for sources must have the specified keys.

Parameters:

Name Type Description Default keys NestedDict

A list (that may contain sub-lists) of required keys.

required

Receives at execution time:

Name Type Description source DbtBouncerSource

The DbtBouncerSourceBase object to check.

Other Parameters (passed via config file):

Name Type Description exclude Optional[str]

Regex pattern to match the source path (i.e the .yml file where the source is configured). Source paths that match the pattern will not be checked.

include Optional[str]

Regex pattern to match the source path (i.e the .yml file where the source is configured). Only source paths that match the pattern will be checked.

severity Optional[Literal['error', 'warn']]

Severity level of the check. Default: error.

Example(s):

manifest_checks:\n    - name: check_source_has_meta_keys\n      keys:\n        - contact:\n            - email\n            - slack\n        - owner\n

Source code in src/dbt_bouncer/checks/manifest/check_sources.py
class CheckSourceHasMetaKeys(BaseCheck):\n    \"\"\"The `meta` config for sources must have the specified keys.\n\n    Parameters:\n        keys (NestedDict): A list (that may contain sub-lists) of required keys.\n\n    Receives:\n        source (DbtBouncerSource): The DbtBouncerSourceBase object to check.\n\n    Other Parameters:\n        exclude (Optional[str]): Regex pattern to match the source path (i.e the .yml file where the source is configured). Source paths that match the pattern will not be checked.\n        include (Optional[str]): Regex pattern to match the source path (i.e the .yml file where the source is configured). Only source paths that match the pattern will be checked.\n        severity (Optional[Literal[\"error\", \"warn\"]]): Severity level of the check. Default: `error`.\n\n    Example(s):\n        ```yaml\n        manifest_checks:\n            - name: check_source_has_meta_keys\n              keys:\n                - contact:\n                    - email\n                    - slack\n                - owner\n        ```\n\n    \"\"\"\n\n    keys: \"NestedDict\"\n    name: Literal[\"check_source_has_meta_keys\"]\n    source: \"DbtBouncerSourceBase\" = Field(default=None)\n\n    def execute(self) -> None:\n        \"\"\"Execute the check.\"\"\"\n        missing_keys = find_missing_meta_keys(\n            meta_config=self.source.meta,\n            required_keys=self.keys.model_dump(),\n        )\n\n        assert (\n            missing_keys == []\n        ), f\"`{self.source.source_name}.{self.source.name}` is missing the following keys from the `meta` config: {[x.replace('>>', '') for x in missing_keys]}\"\n
"},{"location":"checks/manifest/check_sources/#manifest.check_sources.CheckSourceHasTags","title":"CheckSourceHasTags","text":"

Sources must have the specified tags.

Parameters:

Name Type Description Default source DbtBouncerSource

The DbtBouncerSourceBase object to check.

required tags List[str]

List of tags to check for.

required

Other Parameters (passed via config file):

Name Type Description exclude Optional[str]

Regex pattern to match the source path (i.e the .yml file where the source is configured). Source paths that match the pattern will not be checked.

include Optional[str]

Regex pattern to match the source path (i.e the .yml file where the source is configured). Only source paths that match the pattern will be checked.

severity Optional[Literal['error', 'warn']]

Severity level of the check. Default: error.

Example(s):

manifest_checks:\n    - name: check_source_has_tags\n      tags:\n        - tag_1\n        - tag_2\n

Source code in src/dbt_bouncer/checks/manifest/check_sources.py
class CheckSourceHasTags(BaseCheck):\n    \"\"\"Sources must have the specified tags.\n\n    Parameters:\n        source (DbtBouncerSource): The DbtBouncerSourceBase object to check.\n        tags (List[str]): List of tags to check for.\n\n    Other Parameters:\n        exclude (Optional[str]): Regex pattern to match the source path (i.e the .yml file where the source is configured). Source paths that match the pattern will not be checked.\n        include (Optional[str]): Regex pattern to match the source path (i.e the .yml file where the source is configured). Only source paths that match the pattern will be checked.\n        severity (Optional[Literal[\"error\", \"warn\"]]): Severity level of the check. Default: `error`.\n\n    Example(s):\n        ```yaml\n        manifest_checks:\n            - name: check_source_has_tags\n              tags:\n                - tag_1\n                - tag_2\n        ```\n\n    \"\"\"\n\n    name: Literal[\"check_source_has_tags\"]\n    source: \"DbtBouncerSourceBase\" = Field(default=None)\n    tags: List[str]\n\n    def execute(self) -> None:\n        \"\"\"Execute the check.\"\"\"\n        missing_tags = [tag for tag in self.tags if tag not in self.source.tags]\n        assert not missing_tags, f\"`{self.source.source_name}.{self.source.name}` is missing required tags: {missing_tags}.\"\n
"},{"location":"checks/manifest/check_sources/#manifest.check_sources.CheckSourceLoaderPopulated","title":"CheckSourceLoaderPopulated","text":"

Sources must have a populated loader.

Parameters:

Name Type Description Default source DbtBouncerSource

The DbtBouncerSourceBase object to check.

required

Other Parameters (passed via config file):

Name Type Description exclude Optional[str]

Regex pattern to match the source path (i.e the .yml file where the source is configured). Source paths that match the pattern will not be checked.

include Optional[str]

Regex pattern to match the source path (i.e the .yml file where the source is configured). Only source paths that match the pattern will be checked.

severity Optional[Literal['error', 'warn']]

Severity level of the check. Default: error.

Example(s):

manifest_checks:\n    - name: check_source_loader_populated\n

Source code in src/dbt_bouncer/checks/manifest/check_sources.py
class CheckSourceLoaderPopulated(BaseCheck):\n    \"\"\"Sources must have a populated loader.\n\n    Parameters:\n        source (DbtBouncerSource): The DbtBouncerSourceBase object to check.\n\n    Other Parameters:\n        exclude (Optional[str]): Regex pattern to match the source path (i.e the .yml file where the source is configured). Source paths that match the pattern will not be checked.\n        include (Optional[str]): Regex pattern to match the source path (i.e the .yml file where the source is configured). Only source paths that match the pattern will be checked.\n        severity (Optional[Literal[\"error\", \"warn\"]]): Severity level of the check. Default: `error`.\n\n    Example(s):\n        ```yaml\n        manifest_checks:\n            - name: check_source_loader_populated\n        ```\n\n    \"\"\"\n\n    name: Literal[\"check_source_loader_populated\"]\n    source: \"DbtBouncerSourceBase\" = Field(default=None)\n\n    def execute(self) -> None:\n        \"\"\"Execute the check.\"\"\"\n        assert (\n            self.source.loader != \"\"\n        ), f\"`{self.source.source_name}.{self.source.name}` does not have a populated loader.\"\n
"},{"location":"checks/manifest/check_sources/#manifest.check_sources.CheckSourceNames","title":"CheckSourceNames","text":"

Sources must have a name that matches the supplied regex.

Parameters:

Name Type Description Default source_name_pattern str

Regexp the source name must match.

required

Receives at execution time:

Name Type Description source DbtBouncerSource

The DbtBouncerSourceBase object to check.

Other Parameters (passed via config file):

Name Type Description exclude Optional[str]

Regex pattern to match the source path (i.e the .yml file where the source is configured). Source paths that match the pattern will not be checked.

include Optional[str]

Regex pattern to match the source path (i.e the .yml file where the source is configured). Only source paths that match the pattern will be checked.

severity Optional[Literal['error', 'warn']]

Severity level of the check. Default: error.

Example(s):

manifest_checks:\n    - name: check_source_names\n      source_name_pattern: >\n        ^[a-z0-9_]*$\n

Source code in src/dbt_bouncer/checks/manifest/check_sources.py
class CheckSourceNames(BaseCheck):\n    \"\"\"Sources must have a name that matches the supplied regex.\n\n    Parameters:\n        source_name_pattern (str): Regexp the source name must match.\n\n    Receives:\n        source (DbtBouncerSource): The DbtBouncerSourceBase object to check.\n\n    Other Parameters:\n        exclude (Optional[str]): Regex pattern to match the source path (i.e the .yml file where the source is configured). Source paths that match the pattern will not be checked.\n        include (Optional[str]): Regex pattern to match the source path (i.e the .yml file where the source is configured). Only source paths that match the pattern will be checked.\n        severity (Optional[Literal[\"error\", \"warn\"]]): Severity level of the check. Default: `error`.\n\n    Example(s):\n        ```yaml\n        manifest_checks:\n            - name: check_source_names\n              source_name_pattern: >\n                ^[a-z0-9_]*$\n        ```\n\n    \"\"\"\n\n    name: Literal[\"check_source_names\"]\n    source_name_pattern: str\n    source: \"DbtBouncerSourceBase\" = Field(default=None)\n\n    def execute(self) -> None:\n        \"\"\"Execute the check.\"\"\"\n        assert (\n            re.compile(self.source_name_pattern.strip()).match(self.source.name)\n            is not None\n        ), f\"`{self.source.source_name}.{self.source.name}` does not match the supplied regex `({self.source_name_pattern.strip()})`.\"\n
"},{"location":"checks/manifest/check_sources/#manifest.check_sources.CheckSourceNotOrphaned","title":"CheckSourceNotOrphaned","text":"

Sources must be referenced in at least one model.

Receives at execution time:

Name Type Description models List[DbtBouncerModelBase]

List of DbtBouncerModelBase objects parsed from manifest.json.

source DbtBouncerSource

The DbtBouncerSourceBase object to check.

Other Parameters (passed via config file):

Name Type Description exclude Optional[str]

Regex pattern to match the source path (i.e the .yml file where the source is configured). Source paths that match the pattern will not be checked.

include Optional[str]

Regex pattern to match the source path (i.e the .yml file where the source is configured). Only source paths that match the pattern will be checked.

severity Optional[Literal['error', 'warn']]

Severity level of the check. Default: error.

Example(s):

manifest_checks:\n    - name: check_source_not_orphaned\n

Source code in src/dbt_bouncer/checks/manifest/check_sources.py
class CheckSourceNotOrphaned(BaseCheck):\n    \"\"\"Sources must be referenced in at least one model.\n\n    Receives:\n        models (List[DbtBouncerModelBase]): List of DbtBouncerModelBase objects parsed from `manifest.json`.\n        source (DbtBouncerSource): The DbtBouncerSourceBase object to check.\n\n    Other Parameters:\n        exclude (Optional[str]): Regex pattern to match the source path (i.e the .yml file where the source is configured). Source paths that match the pattern will not be checked.\n        include (Optional[str]): Regex pattern to match the source path (i.e the .yml file where the source is configured). Only source paths that match the pattern will be checked.\n        severity (Optional[Literal[\"error\", \"warn\"]]): Severity level of the check. Default: `error`.\n\n    Example(s):\n        ```yaml\n        manifest_checks:\n            - name: check_source_not_orphaned\n        ```\n\n    \"\"\"\n\n    models: List[\"DbtBouncerModelBase\"] = Field(default=[])\n    name: Literal[\"check_source_not_orphaned\"]\n    source: \"DbtBouncerSourceBase\" = Field(default=None)\n\n    def execute(self) -> None:\n        \"\"\"Execute the check.\"\"\"\n        num_refs = sum(\n            self.source.unique_id in model.depends_on.nodes for model in self.models\n        )\n        assert (\n            num_refs >= 1\n        ), f\"Source `{self.source.source_name}.{self.source.name}` is orphaned, i.e. not referenced by any model.\"\n
"},{"location":"checks/manifest/check_sources/#manifest.check_sources.CheckSourcePropertyFileLocation","title":"CheckSourcePropertyFileLocation","text":"

Source properties files must follow the guidance provided by dbt here.

Receives at execution time:

Name Type Description source DbtBouncerSource

The DbtBouncerSourceBase object to check.

Other Parameters (passed via config file):

Name Type Description exclude Optional[str]

Regex pattern to match the source path (i.e the .yml file where the source is configured). Source paths that match the pattern will not be checked.

include Optional[str]

Regex pattern to match the source path (i.e the .yml file where the source is configured). Only source paths that match the pattern will be checked.

severity Optional[Literal['error', 'warn']]

Severity level of the check. Default: error.

Example(s):

manifest_checks:\n    - name: check_source_property_file_location\n

Source code in src/dbt_bouncer/checks/manifest/check_sources.py
class CheckSourcePropertyFileLocation(BaseCheck):\n    \"\"\"Source properties files must follow the guidance provided by dbt [here](https://docs.getdbt.com/best-practices/how-we-structure/1-guide-overview).\n\n    Receives:\n        source (DbtBouncerSource): The DbtBouncerSourceBase object to check.\n\n    Other Parameters:\n        exclude (Optional[str]): Regex pattern to match the source path (i.e the .yml file where the source is configured). Source paths that match the pattern will not be checked.\n        include (Optional[str]): Regex pattern to match the source path (i.e the .yml file where the source is configured). Only source paths that match the pattern will be checked.\n        severity (Optional[Literal[\"error\", \"warn\"]]): Severity level of the check. Default: `error`.\n\n    Example(s):\n        ```yaml\n        manifest_checks:\n            - name: check_source_property_file_location\n        ```\n\n    \"\"\"\n\n    name: Literal[\"check_source_property_file_location\"]\n    source: \"DbtBouncerSourceBase\" = Field(default=None)\n\n    def execute(self) -> None:\n        \"\"\"Execute the check.\"\"\"\n        path_cleaned = clean_path_str(self.source.original_file_path).replace(\n            \"models/staging\", \"\"\n        )\n        expected_substring = \"_\".join(path_cleaned.split(\"/\")[:-1])\n\n        assert path_cleaned.split(\n            \"/\",\n        )[\n            -1\n        ].startswith(\n            \"_\",\n        ), f\"The properties file for `{self.source.source_name}.{self.source.name}` (`{path_cleaned}`) does not start with an underscore.\"\n        assert (\n            expected_substring in path_cleaned\n        ), f\"The properties file for `{self.source.source_name}.{self.source.name}` (`{path_cleaned}`) does not contain the expected substring (`{expected_substring}`).\"\n        assert path_cleaned.split(\n            \"/\",\n        )[\n            -1\n        ].endswith(\n            \"__sources.yml\",\n        ), f\"The properties file for `{self.source.source_name}.{self.source.name}` (`{path_cleaned}`) does not end with `__sources.yml`.\"\n
"},{"location":"checks/manifest/check_sources/#manifest.check_sources.CheckSourceUsedByModelsInSameDirectory","title":"CheckSourceUsedByModelsInSameDirectory","text":"

Sources can only be referenced by models that are located in the same directory where the source is defined.

Parameters:

Name Type Description Default models List[DbtBouncerModelBase]

List of DbtBouncerModelBase objects parsed from manifest.json.

required source DbtBouncerSource

The DbtBouncerSourceBase object to check.

required

Other Parameters (passed via config file):

Name Type Description exclude Optional[str]

Regex pattern to match the source path (i.e the .yml file where the source is configured). Source paths that match the pattern will not be checked.

include Optional[str]

Regex pattern to match the source path (i.e the .yml file where the source is configured). Only source paths that match the pattern will be checked.

severity Optional[Literal['error', 'warn']]

Severity level of the check. Default: error.

Example(s):

manifest_checks:\n    - name: check_source_used_by_models_in_same_directory\n

Source code in src/dbt_bouncer/checks/manifest/check_sources.py
class CheckSourceUsedByModelsInSameDirectory(BaseCheck):\n    \"\"\"Sources can only be referenced by models that are located in the same directory where the source is defined.\n\n    Parameters:\n        models (List[DbtBouncerModelBase]): List of DbtBouncerModelBase objects parsed from `manifest.json`.\n        source (DbtBouncerSource): The DbtBouncerSourceBase object to check.\n\n    Other Parameters:\n        exclude (Optional[str]): Regex pattern to match the source path (i.e the .yml file where the source is configured). Source paths that match the pattern will not be checked.\n        include (Optional[str]): Regex pattern to match the source path (i.e the .yml file where the source is configured). Only source paths that match the pattern will be checked.\n        severity (Optional[Literal[\"error\", \"warn\"]]): Severity level of the check. Default: `error`.\n\n    Example(s):\n        ```yaml\n        manifest_checks:\n            - name: check_source_used_by_models_in_same_directory\n        ```\n\n    \"\"\"\n\n    models: List[\"DbtBouncerModelBase\"] = Field(default=[])\n    name: Literal[\"check_source_used_by_models_in_same_directory\"]\n    source: \"DbtBouncerSourceBase\" = Field(default=None)\n\n    def execute(self) -> None:\n        \"\"\"Execute the check.\"\"\"\n        reffed_models_not_in_same_dir = []\n        for model in self.models:\n            if (\n                self.source.unique_id in model.depends_on.nodes\n                and model.original_file_path.split(\"/\")[:-1]\n                != self.source.original_file_path.split(\"/\")[:-1]\n            ):\n                reffed_models_not_in_same_dir.append(model.unique_id.split(\".\")[0])\n\n        assert (\n            len(reffed_models_not_in_same_dir) == 0\n        ), f\"Source `{self.source.source_name}.{self.source.name}` is referenced by models defined in a different directory: {reffed_models_not_in_same_dir}\"\n
"},{"location":"checks/manifest/check_sources/#manifest.check_sources.CheckSourceUsedByOnlyOneModel","title":"CheckSourceUsedByOnlyOneModel","text":"

Each source can be referenced by a maximum of one model.

Receives at execution time:

Name Type Description models List[DbtBouncerModelBase]

List of DbtBouncerModelBase objects parsed from manifest.json.

source DbtBouncerSource

The DbtBouncerSourceBase object to check.

Other Parameters (passed via config file):

Name Type Description exclude Optional[str]

Regex pattern to match the source path (i.e the .yml file where the source is configured). Source paths that match the pattern will not be checked.

include Optional[str]

Regex pattern to match the source path (i.e the .yml file where the source is configured). Only source paths that match the pattern will be checked.

severity Optional[Literal['error', 'warn']]

Severity level of the check. Default: error.

Example(s):

manifest_checks:\n    - name: check_source_used_by_only_one_model\n

Source code in src/dbt_bouncer/checks/manifest/check_sources.py
class CheckSourceUsedByOnlyOneModel(BaseCheck):\n    \"\"\"Each source can be referenced by a maximum of one model.\n\n    Receives:\n        models (List[DbtBouncerModelBase]): List of DbtBouncerModelBase objects parsed from `manifest.json`.\n        source (DbtBouncerSource): The DbtBouncerSourceBase object to check.\n\n    Other Parameters:\n        exclude (Optional[str]): Regex pattern to match the source path (i.e the .yml file where the source is configured). Source paths that match the pattern will not be checked.\n        include (Optional[str]): Regex pattern to match the source path (i.e the .yml file where the source is configured). Only source paths that match the pattern will be checked.\n        severity (Optional[Literal[\"error\", \"warn\"]]): Severity level of the check. Default: `error`.\n\n    Example(s):\n        ```yaml\n        manifest_checks:\n            - name: check_source_used_by_only_one_model\n        ```\n\n    \"\"\"\n\n    models: List[\"DbtBouncerModelBase\"] = Field(default=[])\n    name: Literal[\"check_source_used_by_only_one_model\"]\n    source: \"DbtBouncerSourceBase\" = Field(default=None)\n\n    def execute(self) -> None:\n        \"\"\"Execute the check.\"\"\"\n        num_refs = sum(\n            self.source.unique_id in model.depends_on.nodes for model in self.models\n        )\n        assert (\n            num_refs <= 1\n        ), f\"Source `{self.source.source_name}.{self.source.name}` is referenced by more than one model.\"\n
"},{"location":"checks/manifest/check_unit_tests/","title":"Manifest Checks: Unit Tests","text":"

Note

The below checks require manifest.json to be present.

"},{"location":"checks/manifest/check_unit_tests/#manifest.check_unit_tests.CheckUnitTestExpectFormats","title":"CheckUnitTestExpectFormats","text":"

Unit tests can only use the specified formats.

Warning

This check is only supported for dbt 1.8.0 and above.

Parameters:

Name Type Description Default permitted_formats Optional[List[Literal['csv', 'dict', 'sql']]]

A list of formats that are allowed to be used for expect input in a unit test.

required

Receives at execution time:

Name Type Description manifest_obj DbtBouncerManifest

The DbtBouncerManifest object parsed from manifest.json.

unit_test UnitTests

The UnitTests object to check.

Other Parameters (passed via config file):

Name Type Description exclude Optional[str]

Regex pattern to match the unit test path (i.e the .yml file where the unit test is configured). Unit test paths that match the pattern will not be checked.

include Optional[str]

Regex pattern to match the unit test path (i.e the .yml file where the unit test is configured). Only unit test paths that match the pattern will be checked.

severity Optional[Literal['error', 'warn']]

Severity level of the check. Default: error.

Example(s):

manifest_checks:\n    - name: check_unit_test_expect_format\n      permitted_formats:\n        - csv\n

Source code in src/dbt_bouncer/checks/manifest/check_unit_tests.py
class CheckUnitTestExpectFormats(BaseCheck):\n    \"\"\"Unit tests can only use the specified formats.\n\n    !!! warning\n\n        This check is only supported for dbt 1.8.0 and above.\n\n    Parameters:\n        permitted_formats (Optional[List[Literal[\"csv\", \"dict\", \"sql\"]]]): A list of formats that are allowed to be used for `expect` input in a unit test.\n\n    Receives:\n        manifest_obj (DbtBouncerManifest): The DbtBouncerManifest object parsed from `manifest.json`.\n        unit_test (UnitTests): The UnitTests object to check.\n\n    Other Parameters:\n        exclude (Optional[str]): Regex pattern to match the unit test path (i.e the .yml file where the unit test is configured). Unit test paths that match the pattern will not be checked.\n        include (Optional[str]): Regex pattern to match the unit test path (i.e the .yml file where the unit test is configured). Only unit test paths that match the pattern will be checked.\n        severity (Optional[Literal[\"error\", \"warn\"]]): Severity level of the check. Default: `error`.\n\n    Example(s):\n        ```yaml\n        manifest_checks:\n            - name: check_unit_test_expect_format\n              permitted_formats:\n                - csv\n        ```\n\n    \"\"\"\n\n    manifest_obj: \"DbtBouncerManifest\" = Field(default=None)\n    name: Literal[\"check_unit_test_expect_format\"]\n    permitted_formats: List[Literal[\"csv\", \"dict\", \"sql\"]] = Field(\n        default=[\"csv\", \"dict\", \"sql\"],\n    )\n    unit_test: \"UnitTests\" = Field(default=None)\n\n    def execute(self) -> None:\n        \"\"\"Execute the check.\"\"\"\n        if (\n            semver.Version.parse(self.manifest_obj.manifest.metadata.dbt_version)\n            >= \"1.8.0\"\n        ):\n            assert (\n                self.unit_test.expect.format.value in self.permitted_formats\n            ), f\"Unit test `{self.unit_test.name}` has an `expect` format that is not permitted. Permitted formats are: {self.permitted_formats}.\"\n        else:\n            logging.warning(\n                \"The `check_unit_test_expect_format` check is only supported for dbt 1.8.0 and above.\",\n            )\n
"},{"location":"checks/manifest/check_unit_tests/#manifest.check_unit_tests.CheckUnitTestGivenFormats","title":"CheckUnitTestGivenFormats","text":"

Unit tests can only use the specified formats.

Warning

This check is only supported for dbt 1.8.0 and above.

Parameters:

Name Type Description Default permitted_formats Optional[List[Literal['csv', 'dict', 'sql']]]

A list of formats that are allowed to be used for expect input in a unit test.

required

Receives at execution time:

Name Type Description manifest_obj DbtBouncerManifest

The DbtBouncerManifest object parsed from manifest.json.

unit_test UnitTests

The UnitTests object to check.

Other Parameters (passed via config file):

Name Type Description exclude Optional[str]

Regex pattern to match the unit test path (i.e the .yml file where the unit test is configured). Unit test paths that match the pattern will not be checked.

include Optional[str]

Regex pattern to match the unit test path (i.e the .yml file where the unit test is configured). Only unit test paths that match the pattern will be checked.

severity Optional[Literal['error', 'warn']]

Severity level of the check. Default: error.

Example(s):

manifest_checks:\n    - name: check_unit_test_given_formats\n      permitted_formats:\n        - csv\n

Source code in src/dbt_bouncer/checks/manifest/check_unit_tests.py
class CheckUnitTestGivenFormats(BaseCheck):\n    \"\"\"Unit tests can only use the specified formats.\n\n    !!! warning\n\n        This check is only supported for dbt 1.8.0 and above.\n\n    Parameters:\n        permitted_formats (Optional[List[Literal[\"csv\", \"dict\", \"sql\"]]]): A list of formats that are allowed to be used for `expect` input in a unit test.\n\n    Receives:\n        manifest_obj (DbtBouncerManifest): The DbtBouncerManifest object parsed from `manifest.json`.\n        unit_test (UnitTests): The UnitTests object to check.\n\n    Other Parameters:\n        exclude (Optional[str]): Regex pattern to match the unit test path (i.e the .yml file where the unit test is configured). Unit test paths that match the pattern will not be checked.\n        include (Optional[str]): Regex pattern to match the unit test path (i.e the .yml file where the unit test is configured). Only unit test paths that match the pattern will be checked.\n        severity (Optional[Literal[\"error\", \"warn\"]]): Severity level of the check. Default: `error`.\n\n    Example(s):\n        ```yaml\n        manifest_checks:\n            - name: check_unit_test_given_formats\n              permitted_formats:\n                - csv\n        ```\n\n    \"\"\"\n\n    manifest_obj: \"DbtBouncerManifest\" = Field(default=None)\n    name: Literal[\"check_unit_test_given_formats\"]\n    permitted_formats: List[Literal[\"csv\", \"dict\", \"sql\"]] = Field(\n        default=[\"csv\", \"dict\", \"sql\"],\n    )\n    unit_test: \"UnitTests\" = Field(default=None)\n\n    def execute(self) -> None:\n        \"\"\"Execute the check.\"\"\"\n        if (\n            semver.Version.parse(self.manifest_obj.manifest.metadata.dbt_version)\n            >= \"1.8.0\"\n        ):\n            given_formats = [i.format.value for i in self.unit_test.given]\n            assert all(\n                e in self.permitted_formats for e in given_formats\n            ), f\"Unit test `{self.unit_test.name}` has given formats which are not permitted. Permitted formats are: {self.permitted_formats}.\"\n        else:\n            logging.warning(\n                \"The `check_unit_test_given_formats` check is only supported for dbt 1.8.0 and above.\",\n            )\n
"},{"location":"checks/run_results/check_run_results/","title":"Run Results Checks","text":"

Note

The below checks require both manifest.json and run_results.json to be present.

"},{"location":"checks/run_results/check_run_results/#run_results.check_run_results.CheckRunResultsMaxGigabytesBilled","title":"CheckRunResultsMaxGigabytesBilled","text":"

Each result can have a maximum number of gigabytes billed.

Note

Note that this check only works for the dbt-bigquery adapter.

Parameters:

Name Type Description Default max_gigabytes_billed float

The maximum number of gigabytes billed.

required run_result DbtBouncerRunResult

The DbtBouncerRunResult object to check.

required

Other Parameters (passed via config file):

Name Type Description exclude Optional[str]

Regex pattern to match the resource path. Resource paths that match the pattern will not be checked.

include Optional[str]

Regex pattern to match the resource path. Only resource paths that match the pattern will be checked.

severity Optional[Literal['error', 'warn']]

Severity level of the check. Default: error.

Example(s):

run_results_checks:\n    - name: check_run_results_max_gigabytes_billed\n    max_gigabytes_billed: 100\n

Source code in src/dbt_bouncer/checks/run_results/check_run_results.py
class CheckRunResultsMaxGigabytesBilled(BaseCheck):\n    \"\"\"Each result can have a maximum number of gigabytes billed.\n\n    !!! note\n\n        Note that this check only works for the `dbt-bigquery` adapter.\n\n    Parameters:\n        max_gigabytes_billed (float): The maximum number of gigabytes billed.\n        run_result (DbtBouncerRunResult): The DbtBouncerRunResult object to check.\n\n    Other Parameters:\n        exclude (Optional[str]): Regex pattern to match the resource path. Resource paths that match the pattern will not be checked.\n        include (Optional[str]): Regex pattern to match the resource path. Only resource paths that match the pattern will be checked.\n        severity (Optional[Literal[\"error\", \"warn\"]]): Severity level of the check. Default: `error`.\n\n    Raises: # noqa:DOC502\n        KeyError: If the `dbt-bigquery` adapter is not used.\n\n    Example(s):\n        ```yaml\n        run_results_checks:\n            - name: check_run_results_max_gigabytes_billed\n            max_gigabytes_billed: 100\n        ```\n\n    \"\"\"\n\n    max_gigabytes_billed: float\n    name: Literal[\"check_run_results_max_gigabytes_billed\"]\n    run_result: Optional[\"DbtBouncerRunResultBase\"] = Field(default=None)\n\n    def execute(self) -> None:\n        \"\"\"Execute the check.\"\"\"\n        try:\n            gigabytes_billed = self.run_result.adapter_response[\"bytes_billed\"] / (\n                1000**3\n            )\n        except KeyError as e:\n            raise RuntimeError(  # noqa: DOC501\n                \"`bytes_billed` not found in adapter response. Are you using the `dbt-bigquery` adapter?\",\n            ) from e\n\n        assert (\n            gigabytes_billed < self.max_gigabytes_billed\n        ), f\"`{self.run_result.unique_id.split('.')[-2]}` results in ({gigabytes_billed} billed bytes, this is greater than permitted ({self.max_gigabytes_billed}).\"\n
"},{"location":"checks/run_results/check_run_results/#run_results.check_run_results.CheckRunResultsMaxExecutionTime","title":"CheckRunResultsMaxExecutionTime","text":"

Each result can take a maximum duration (seconds).

Parameters:

Name Type Description Default max_execution_time_seconds float

The maximum execution time (seconds) allowed for a node.

required

Receives at execution time:

Name Type Description run_result DbtBouncerRunResult

The DbtBouncerRunResult object to check.

Other Parameters (passed via config file):

Name Type Description exclude Optional[str]

Regex pattern to match the resource path. Resource paths that match the pattern will not be checked.

include Optional[str]

Regex pattern to match the resource path. Only resource paths that match the pattern will be checked.

severity Optional[Literal['error', 'warn']]

Severity level of the check. Default: error.

Example(s):

run_results_checks:\n    - name: check_run_results_max_execution_time\n      max_execution_time_seconds: 60\n
run_results_checks:\n    - name: check_run_results_max_execution_time\n      include: ^models/staging # Not a good idea, here for demonstration purposes only\n      max_execution_time_seconds: 10\n

Source code in src/dbt_bouncer/checks/run_results/check_run_results.py
class CheckRunResultsMaxExecutionTime(BaseCheck):\n    \"\"\"Each result can take a maximum duration (seconds).\n\n    Parameters:\n        max_execution_time_seconds (float): The maximum execution time (seconds) allowed for a node.\n\n    Receives:\n        run_result (DbtBouncerRunResult): The DbtBouncerRunResult object to check.\n\n    Other Parameters:\n        exclude (Optional[str]): Regex pattern to match the resource path. Resource paths that match the pattern will not be checked.\n        include (Optional[str]): Regex pattern to match the resource path. Only resource paths that match the pattern will be checked.\n        severity (Optional[Literal[\"error\", \"warn\"]]): Severity level of the check. Default: `error`.\n\n    Example(s):\n        ```yaml\n        run_results_checks:\n            - name: check_run_results_max_execution_time\n              max_execution_time_seconds: 60\n        ```\n        ```yaml\n        run_results_checks:\n            - name: check_run_results_max_execution_time\n              include: ^models/staging # Not a good idea, here for demonstration purposes only\n              max_execution_time_seconds: 10\n        ```\n\n    \"\"\"\n\n    max_execution_time_seconds: float\n    name: Literal[\"check_run_results_max_execution_time\"]\n    run_result: Optional[\"DbtBouncerRunResultBase\"] = Field(default=None)\n\n    def execute(self) -> None:\n        \"\"\"Execute the check.\"\"\"\n        assert (\n            self.run_result.execution_time <= self.max_execution_time_seconds\n        ), f\"`{self.run_result.unique_id.split('.')[-1]}` has an execution time ({self.run_result.execution_time} greater than permitted ({self.max_execution_time_seconds}s).\"\n
"}]} \ No newline at end of file +{"config":{"lang":["en"],"separator":"[\\s\\-]+","pipeline":["stopWordFilter"]},"docs":[{"location":"","title":"Home","text":""},{"location":"#welcome-to-dbt-bouncer","title":"Welcome to dbt-bouncer","text":"

dbt-bouncer is an open-source tool that allows you to configure and enforce conventions for your dbt project. The conventions are run against dbt's artifact files (think ./target/manifest.json) resulting in speedy tests. Conventions can be specified in a .yml file, allowing maximum customisation to the conventions you wish to follow (or create \ud83d\ude00).

Check out our Getting Started guide.

"},{"location":"#terminology","title":"Terminology","text":"
  • Check: A check is a rule run against a dbt artifact.
  • Config file: A .yml file that specifies which checks to run along with any parameters.
  • dbt artifacts directory: The directory that contains the dbt artifacts (manifest.json, etc.), generally this is ./target.
"},{"location":"#aims","title":"Aims","text":"

dbt-bouncer aims to:

  • Provide a 100% configurable way to enforce conventions in a dbt project.
  • Be as fast as possible, running checks against dbt artifacts.
  • Be as easy as possible to use, with a simple config file written in YML.
  • Be as flexible as possible, allowing checks to be written in python.
  • Provide immediate feedback when run as part of a CI pipeline.
"},{"location":"#about","title":"About","text":"

dbt-bouncer is free software, released under the MIT license. It originated at Xebia Data in Amsterdam, Netherlands. Source code is available on GitHub.

All contributions, in the form of bug reports, pull requests, feedback or discussion are welcome. See the contributing guide for more information.

"},{"location":"CONTRIBUTING/","title":"Contributing to dbt-bouncer","text":"

dbt-bouncer is open source software. Whether you are a seasoned open source contributor or a first-time committer, we welcome and encourage you to contribute code, documentation, ideas, or problem statements to this project.

"},{"location":"CONTRIBUTING/#about-this-document","title":"About this document","text":"

There are many ways to contribute to the ongoing development of dbt-bouncer, such as by participating in discussions and issues.

The rest of this document serves as a more granular guide for contributing code changes to dbt-bouncer (this repository). It is not intended as a guide for using dbt-bouncer, and some pieces assume a level of familiarity with Python development (virtualenvs, Poetry, etc). Specific code snippets in this guide assume you are using macOS or Linux and are comfortable with the command line.

If you get stuck, we're happy to help! Just open an issue or draft PR and we'll do our best to help out.

"},{"location":"CONTRIBUTING/#note","title":"Note","text":"
  • Branches: All pull requests from community contributors should target the main branch (default).
"},{"location":"CONTRIBUTING/#getting-the-code","title":"Getting the code","text":""},{"location":"CONTRIBUTING/#installing-git","title":"Installing git","text":"

You will need git in order to download and modify the dbt-bouncer source code. On macOS, the best way to download git is to just install Xcode.

"},{"location":"CONTRIBUTING/#contributors","title":"Contributors","text":"

You can contribute to dbt-bouncer by forking the dbt-bouncer repository. For a detailed overview on forking, check out the GitHub docs on forking. In short, you will need to:

  1. Fork the dbt-bouncer repository.
  2. Clone your fork locally.
  3. Check out a new branch for your proposed changes.
  4. Push changes to your fork.
  5. Open a pull request against godatadriven/dbt-bouncer from your forked repository.
"},{"location":"CONTRIBUTING/#setting-up-an-environment","title":"Setting up an environment","text":"

There are some tools that will be helpful to you in developing locally. While this is the list relevant for dbt-bouncer development, many of these tools are used commonly across open-source python projects.

"},{"location":"CONTRIBUTING/#tools","title":"Tools","text":"

These are the tools used in dbt-bouncer development and testing:

  • click to create our CLI interface.
  • GitHub Actions for automating tests and checks, once a PR is pushed to the dbt-bouncer repository.
  • make to run multiple setup or test steps in combination.
  • mypy for static type checking.
  • Poetry to manage our python virtual environment.
  • pre-commit to easily run those checks.
  • Pydantic to validate our configuration file.
  • pytest to define, discover, and run tests.
  • Ruff to lint and format python code.

A deep understanding of these tools in not required to effectively contribute to dbt-bouncer, but we recommend checking out the attached documentation if you're interested in learning more about each one.

"},{"location":"CONTRIBUTING/#virtual-environments","title":"Virtual environments","text":"

We strongly recommend using virtual environments when developing code in dbt-bouncer. We recommend creating this virtualenv in the root of the dbt-bouncer repository. To create a new virtualenv, run:

poetry shell\n

This will create a new Python virtual environment.

"},{"location":"CONTRIBUTING/#setting-environment-variables","title":"Setting environment variables","text":"

Set required environment variables by copying .env.example to .env and updating the values.

"},{"location":"CONTRIBUTING/#running-dbt-bouncer-in-development","title":"Running dbt-bouncer in development","text":""},{"location":"CONTRIBUTING/#installation","title":"Installation","text":"

First make sure that you set up your virtualenv as described in Setting up an environment. Next, install dbt-bouncer, its dependencies and pre-commit:

poetry install\npoetry run pre-commit install\n

When installed in this way, any changes you make to your local copy of the source code will be reflected immediately in your next dbt-bouncer run.

"},{"location":"CONTRIBUTING/#running-dbt-bouncer","title":"Running dbt-bouncer","text":"

With your virtualenv activated, the dbt-bouncer script should point back to the source code you've cloned on your machine. You can verify this by running which dbt-bouncer. This command should show you a path to an executable in your virtualenv. You can run dbt-bouncer using the provided example configuration file via:

poetry run dbt-bouncer --config-file dbt-bouncer-example.yml\n
"},{"location":"CONTRIBUTING/#testing","title":"Testing","text":"

Once you're able to manually test that your code change is working as expected, it's important to run existing automated tests, as well as adding some new ones. These tests will ensure that: - Your code changes do not unexpectedly break other established functionality - Your code changes can handle all known edge cases - The functionality you're adding will keep working in the future

"},{"location":"CONTRIBUTING/#note_1","title":"Note","text":"
  • Generating dbt artifacts: If you change the configuration of the dbt project located in dbt_project then you will need to re-generate the dbt artifacts used in testing. To do so, run:
make build-artifacts\n
"},{"location":"CONTRIBUTING/#test-commands","title":"Test commands","text":"

There are a few methods for running tests locally.

"},{"location":"CONTRIBUTING/#makefile","title":"makefile","text":"

There are multiple targets in the makefile to run common test suites, most notably:

# Runs unit tests\nmake test-unit\n\n# Runs integration tests\nmake test-integration\n\n# Runs all tests\nmake test\n
"},{"location":"CONTRIBUTING/#pre-commit","title":"pre-commit","text":"

pre-commit takes care of running all code-checks for formatting and linting. Run poetry run pre-commit install to install pre-commit in your local environment. Once this is done you can use the git pre-commit hooks to ensure proper formatting and linting.

"},{"location":"CONTRIBUTING/#pytest","title":"pytest","text":"

Finally, you can also run a specific test or group of tests using pytest directly. With a virtualenv active and dev dependencies installed you can do things like:

# run all unit tests in a file\npoetry run pytest ./tests/unit/checks/catalog/test_columns.py\n\n# run a specific unit test\npoetry run pytest ./tests/unit/checks/catalog/test_columns.py::test_check_columns_are_documented_in_public_models\n

See pytest usage docs for an overview of useful command-line options.

"},{"location":"CONTRIBUTING/#assorted-development-tips","title":"Assorted development tips","text":"
  • Append # type: ignore to the end of a line if you need to disable mypy on that line, preferably with the specific rule to ignore such as # type: ignore[union-attr].
"},{"location":"CONTRIBUTING/#adding-a-new-check","title":"Adding a new check","text":"

To add a new check follow the below steps:

  1. In ./src/dbt_bouncer/checks choose the appropriate directory for your check. For example, if your check only requires the manifest.json then use the manifest directory, if your check requires the catalog.json then use the catalog directory.
  2. Within the chosen directory assess if a suitable file already exists. For example, if your check applies to a model then manifest/check_models.py is a suitable location.
  3. Within the chosen file, add a Pydantic model, this object must meet the following criteria:

    • Start with \"Check\".
    • Inherit from dbt_bouncer.check_base.BaseCheck.
    • Have a name attribute that is a string whose value is the snake case equivalent of the class name.
    • A default value provided for optional input arguments and arguments that are received at execution time.
    • Have a doc string that includes a description of the check, a list of possible input parameters and at least one example.
    • A clear message in the event of a failure.
  4. After the check is added, add the check to dbt-bouncer-example.yml and run dbt-bouncer --config-file dbt-bouncer-example.yml to ensure the check succeeds.

  5. (Optional) If the dbt project located in ./dbt_project needs to be updated then do so and also run make build-artifacts to generate the new test artifacts.
  6. Add at least one happy path and one unhappy path test to ./tests. The appropriate test file will be the one matching the directory of the check. For example, if the check is in ./src/dbt_bouncer/checks/catalog/check_columns.py then the test file will be ./tests/unit/checks/catalog/test_columns.py.
  7. Run make test to ensure the tests pass.
  8. Open a PR \ud83c\udf89!
"},{"location":"CONTRIBUTING/#submitting-a-pull-request","title":"Submitting a Pull Request","text":"

Code can be merged into the current development branch main by opening a pull request. If the proposal looks like it's on the right track, then a dbt-bouncer maintainer will review the PR. They may suggest code revision for style or clarity, or request that you add unit or integration test(s). These are good things! We believe that, with a little bit of help, anyone can contribute high-quality code. Once merged, your contribution will be available for the next release of dbt-bouncer.

Automated tests run via GitHub Actions. If you're a first-time contributor, all tests will require a maintainer to approve.

Once all tests are passing and your PR has been approved, a dbt-bouncer maintainer will merge your changes into the active development branch. And that's it! Happy developing :tada:

"},{"location":"cli/","title":"CLI","text":"

This page provides documentation for the dbt-bouncer CLI.

"},{"location":"cli/#dbt-bouncer","title":"dbt-bouncer","text":"

Entrypoint for dbt-bouncer.

Raises: RuntimeError: If output file has an invalid extension.

Usage:

dbt-bouncer [OPTIONS]\n

Options:

  --config-file PUREPATH  Location of the YML config file.\n  --output-file PATH      Location of the json file where check metadata will\n                          be saved.\n  -v, --verbosity         Verbosity.\n  --version               Show the version and exit.\n  --help                  Show this message and exit.\n
"},{"location":"cli/#exit-codes","title":"Exit codes","text":"

dbt-bouncer returns the following exit codes:

  • 0: All checks have succeeded.

  • 1:

    • At least one check has failed. Check the logs for more information.
    • A fatal error occurred during check setup or check execution. Check the logs for more information.
"},{"location":"config_file/","title":"Config file","text":"

dbt-bouncer requires a config file which determines what checks are run. The following options are available, in order of priority:

  1. A file passed via the --config-file CLI flag.
  2. A file named dbt-bouncer.yml in the current working directory.
  3. A [tool.dbt-bouncer] section in pyproject.toml.

Here is an example config file in yaml:

# [Optional] Directory where the dbt artifacts exists, generally the `target` directory inside a dbt project. Defaults to `./target`.\ndbt_artifacts_dir: target\n\nmanifest_checks:\n  - name: check_macro_name_matches_file_name\n  - name: check_model_names\n    include: ^models/staging\n    model_name_pattern: ^stg_\n

And the same config in toml:

[tool.dbt-bouncer]\n# [Optional] Directory where the dbt artifacts exists, generally the `target` directory inside a dbt project. Defaults to `./target`.\ndbt_artifacts_dir = \"target\"\n\n[[tool.dbt-bouncer.manifest_checks]]\nname = \"check_macro_name_matches_file_name\"\n\n[[tool.dbt-bouncer.manifest_checks]]\nname = \"check_model_names\"\ninclude = \"^models/staging\"\nmodel_name_pattern = \"^stg_\"\n

For more example config files, see here.

"},{"location":"config_file/#common-arguments","title":"Common arguments","text":""},{"location":"config_file/#exclude-and-include","title":"Exclude and Include","text":"

Most (but not all) checks accept the following optional arguments:

  • exclude: Regexp to match which original file paths to exclude.
  • include: Regexp to match which original file paths to include.

Example per resource type:

  • Exposures: The original file path to the properties file where the source is defined, e.g. ^models/marts/finance will match exposures defined in ./models/marts/finance/_exposures.yml.
  • Macros: The original file path to the macro file, e.g. ^macros/system will match files like ./macros/system/generate_schema_name.sql.
  • Models: The original file path to the model file, e.g. ^marts will match files like ./models/marts/customers.sql.
  • Run results: The original file path to the file associated with the resource, e.g. ^seeds/finance will match seeds in ./seeds/finance, ^models/staging will match models and tests in ./models/staging.
  • Semantic models: The original file path to the properties file where the semantic model is defined, e.g. ^models/marts/finance will match semantic models defined in ./models/marts/finance/_finance__semantic_models.yml.
  • Sources: The original file path to the properties file where the source is defined, e.g. ^models/staging/crm will match sources defined in ./models/staging/crm/_crm__sources.yml.
  • Unit tests: The original file path to the properties file where the unit test is defined, e.g. ^models/staging/crm will match unit tests defined in ^staging/crm/_stg_crm__unit_tests.yml.

To determine if a check accepts these arguments view the Checks page.

Note

exclude and include can be specified at both the check level and the global level. Should both levels be specified, then the check level is applied. All the below examples result in the check_model_names check being run on all models in ./models/staging:

# Specify `include` at the check level only\nmanifest_checks:\n  - name: check_model_names\n    include: ^models/staging\n    model_name_pattern: ^stg_\n
# Specify `include` at the check and global levels\ninclude: ^models/marts\nmanifest_checks:\n  - name: check_model_names\n    include: ^models/staging\n    model_name_pattern: ^stg_\n
# Specify `include` at the global level only\ninclude: ^models/staging\nmanifest_checks:\n  - name: check_model_names\n    model_name_pattern: ^stg_\n

Note

When compiled on Windows machines, keys such as original_file_path, patch_path and path take the form:

models\\\\staging\\\\crm\\\\model_1.sql\n

When compiled on Linux and Mac machines, these same keys take the form:

models/staging/crm/model_1.sql\n

dbt-bouncer converts all of these paths to the Linux/Mac form, hence when you are supplying values to exclude and include you should use the Linux/Mac form.

"},{"location":"config_file/#severity","title":"Severity","text":"

All checks accept a severity argument, valid values are:

  • error: If the check fails then dbt-bouncer will return a non-zero exit code.
  • warn: If the check fails then dbt-bouncer will return a non-zero exit code.

severity can also be specified globally, this is useful when applying dbt-bouncer to a pre-existing dbt project. It allows you to run dbt-bouncer, identify the checks that fail and address the failures in your own time without receiving non-zero exit codes:

# Specify `severity` at the global levels: all checks will have a `warn` severity, avoiding non-zero exit codes.\nseverity: warn\n\nmanifest_checks:\n  - name: check_exposure_based_on_view\n  ...\n

Note

severity can be specified at both the check level and the global level. Should both levels be specified, then the global level is applied.

# No `severity` specified: check will have an `error` severity.\nmanifest_checks:\n  - name: check_exposure_based_on_view\n
# Specify `severity` at the check level only: check will have a `warn` severity.\nmanifest_checks:\n  - name: check_exposure_based_on_view\n    severity: warn\n
# Specify `severity` at the check and global levels: check will have a `warn` severity.\nseverity: warn\nmanifest_checks:\n  - name: check_exposure_based_on_view\n    severity: error\n
# Specify `severity` at the global level only: check will have a `warn` severity.\nseverity: warn\nmanifest_checks:\n  - name: check_exposure_based_on_view\n
"},{"location":"faq/","title":"Frequently Asked Questions","text":""},{"location":"faq/#can-other-tools-perform-the-same-checks-as-dbt-bouncer","title":"Can other tools perform the same checks as dbt-bouncer?","text":"

There are several other tools that perform similar tasks as dbt-bouncer.

  • dbt-checkpoint: A collection of pre-commit hooks for dbt projects. Tests are written in python. Configuration is performed via .pre-commit-config.yaml. Provided the dbt artifacts have already been generated, dbt-checkpoint does not need access to the underlying database. The hooks execute when a new commit is made, as such dbt-checkpoint is designed to be run only as part of pre-commit.
  • dbt-project-evaluator: This is a dbt package from dbt Labs. Tests are written in .sql files using a combination of Jinja and SQL. Configuration is performed via dbt_project.yml and seed files (i.e. csv files). Requires a connection to underlying database. Designed to be run both in a CI pipeline and also during active development.
  • dbt-score: This is a python package installable via pip. A collection of tests that apply only to dbt models. Tests can be executed from the command line. Tests are written in python. Configuration is performed via a pyproject.toml file. Provided the dbt artifacts have already been generated, dbt-score does not need access to the underlying database. Designed to be run during development.

While the above tools inhabit the same space as dbt-bouncer they do not provide what we consider to be the optimum experience that dbt-bouncer provides:

  • Designed to run both locally and in a CI pipeline.
  • Configurable via a file format, YML, that dbt developers are already familiar with.
  • Does not require database access.
  • Can run tests against any of dbt's artifacts.
  • Allows tests to be written in python.

As such we consider dbt-bouncer to be the best tool to enforce conventions in a dbt project.

Tip

dbt-bouncer can perform all the tests currently included in dbt-checkpoint, dbt-project-evaluator and dbt-score. If you see an existing test that is not possible with dbt-bouncer, open an issue and we'll add it!

"},{"location":"faq/#how-to-set-up-dbt-bouncer-in-a-monorepo","title":"How to set up dbt-bouncer in a monorepo?","text":"

A monorepo may consist of one directory with a dbt project and other directories with unrelated code. It may be desired for dbt-bouncer to be configured from the root directory. Sample directory tree:

.\n\u251c\u2500\u2500 dbt-bouncer.yml\n\u251c\u2500\u2500 README.md\n\u251c\u2500\u2500 dbt-project\n\u2502   \u251c\u2500\u2500 models\n\u2502   \u251c\u2500\u2500 dbt_project.yml\n\u2502   \u2514\u2500\u2500 profiles.yml\n\u2514\u2500\u2500 package-a\n    \u251c\u2500\u2500 src\n    \u251c\u2500\u2500 tests\n    \u2514\u2500\u2500 package.json\n

To ease configuration you can use exclude or include at the global level (see Config File for more details). For the above example dbt-bouncer.yml could be configured as:

dbt_artifacts_dir: dbt-project/target\ninclude: ^dbt-project\n\nmanifest_checks:\n    - name: check_exposure_based_on_non_public_models\n

dbt-bouncer can now be run from the root directory.

"},{"location":"getting_started/","title":"Getting Started","text":""},{"location":"getting_started/#how-to-run-dbt-bouncer","title":"How to run dbt-bouncer","text":"
  1. Generate dbt artifacts by running a dbt command:

    • dbt parse to generate a manifest.json artifact.
    • dbt docs generate to generate a catalog.json artifact (necessary if you are using catalog checks).
    • dbt run (or any other command that implies it e.g. dbt build) to generate a run_results.json artifact (necessary if you are using run results checks).
  2. Create a dbt-bouncer.yml config file, details here.

  3. Run dbt-bouncer to validate that your conventions are being maintained.

"},{"location":"getting_started/#installing-with-python","title":"Installing with Python","text":"

Install from pypi.org:

pip install dbt-bouncer # or via any other package manager\n

Run:

dbt-bouncer --config-file <PATH_TO_CONFIG_FILE>\n
Running dbt-bouncer (X.X.X)...\nLoaded config from dbt-bouncer-example.yml...\nValidating conf...\n

dbt-bouncer also supports a verbose mode, run:

dbt-bouncer --config-file <PATH_TO_CONFIG_FILE> -v\n
Running dbt-bouncer (X.X.X)...\nconfig_file=PosixPath('dbt-bouncer-example.yml')\nconfig_file_source='COMMANDLINE'\nConfig file passed via command line: dbt-bouncer-example.yml\nLoading config from /home/pslattery/repos/dbt-bouncer/dbt-bouncer-example.yml...\nLoading config from dbt-bouncer-example.yml...\nLoaded config from dbt-bouncer-example.yml...\nconf={'dbt_artifacts_dir': 'dbt_project/target', 'catalog_checks': [{'name': 'check_column_name_complies_to_column_type', 'column_name_pattern': '^is_.*', 'exclude': '^staging', 'types': ['BOOLEAN']}]}\nValidating conf...\n
"},{"location":"getting_started/#running-as-an-executable-using-uv","title":"Running as an executable using uv","text":"

Run dbt-bouncer as a standalone Python executable using uv:

uvx dbt-bouncer --config-file <PATH_TO_CONFIG_FILE>\n
"},{"location":"getting_started/#github-actions","title":"GitHub Actions","text":"

Run dbt-bouncer as part of your CI pipeline:

name: CI pipeline\n\non:\n  pull_request:\n      branches:\n          - main\n\njobs:\n    run-dbt-bouncer:\n        permissions:\n            pull-requests: write # Required to write a comment on the PR\n        runs-on: ubuntu-latest\n        steps:\n            - name: Checkout\n              uses: actions/checkout@v4\n\n            - name: Generate or fetch dbt artifacts\n              run: ...\n\n            - uses: godatadriven/dbt-bouncer@vX.X\n              with:\n                config-file: ./<PATH_TO_CONFIG_FILE>\n                output-file: results.json # optional, default does not save a results file\n                send-pr-comment: true # optional, defaults to true\n                verbose: false # optional, defaults to false\n

We recommend pinning both a major and minor version number.

"},{"location":"getting_started/#docker","title":"Docker","text":"

Run dbt-bouncer via Docker:

docker run --rm \\\n    --volume \"$PWD\":/app \\\n    ghcr.io/godatadriven/dbt-bouncer:vX.X.X \\\n    --config-file /app/<PATH_TO_CONFIG_FILE>\n
"},{"location":"getting_started/#pex","title":"Pex","text":"

You can also run the .pex (Python EXecutable) artifact directly once you have a python executable (3.8 -> 3.12) installed:

wget https://github.com/godatadriven/dbt-bouncer/releases/download/vX.X.X/dbt-bouncer.pex -O dbt-bouncer.pex\n\npython dbt-bouncer.pex --config-file $PWD/<PATH_TO_CONFIG_FILE>\n
"},{"location":"getting_started/#how-to-contribute-a-check-to-dbt-bouncer","title":"How to contribute a check to dbt-bouncer","text":"

See Adding a new check.

"},{"location":"getting_started/#how-to-add-a-custom-check-to-dbt-bouncer","title":"How to add a custom check to dbt-bouncer","text":"

In addition to the checks built into dbt-bouncer, the ability to add custom checks is supported. This allows users to write checks that are specific to the conventions of their projects. To add a custom check:

  1. Create an empty directory and add a custom_checks_dir key to your config file. The value of this key should be the path to the directory you just created, relative to where the config file is located.
  2. In this directory create an empty __init__.py file.
  3. In this directory create a subdirectory named catalog, manifest or run_results depending on the type of artifact you want to check.
  4. In this subdirectory create a python file that defines a check. The check must meet the following criteria:

    • Start with \"Check\".
    • Inherit from dbt_bouncer.check_base.BaseCheck.
    • Have a name attribute that is a string whose value is the snake case equivalent of the class name.
    • A default value provided for optional input arguments and arguments that are received at execution time.
    • Have a doc string that includes a description of the check, a list of possible input parameters and at least one example.
    • A clear message in the event of a failure.
  5. In your config file, add the name of the check and any desired arguments.

  6. Run dbt-bouncer, your custom check will be executed.

An example:

  • Directory tree:

    .\n\u251c\u2500\u2500 dbt-bouncer.yml\n\u251c\u2500\u2500 dbt_project.yml\n\u251c\u2500\u2500 my_custom_checks\n|   \u251c\u2500\u2500 __init__.py\n|   \u2514\u2500\u2500 manifest\n|       \u2514\u2500\u2500 check_custom_to_me.py\n\u2514\u2500\u2500 target\n    \u2514\u2500\u2500 manifest.json\n
  • Contents of check_custom_to_me.py:

    from typing import TYPE_CHECKING, Literal\n\nfrom pydantic import Field\n\nfrom dbt_bouncer.check_base import BaseCheck\n\nif TYPE_CHECKING:\n    import warnings\n\n    with warnings.catch_warnings():\n        warnings.filterwarnings(\"ignore\", category=UserWarning)\n        from dbt_bouncer.parsers import DbtBouncerModelBase\n\n\nclass CheckModelDepcrecationDate(BaseCheck):\n\n    model: \"DbtBouncerModelBase\" = Field(default=None)\n    name: Literal[\"check_model_deprecation_date\"]\n\n    def execute(self) -> None:\n        \"\"\"Execute the check.\"\"\"\n\n        assert self.model.deprecation_date is not None, f\"`{self.model.name}` requires a `deprecation_date` to be set.\"\n
  • Contents of dbt-bouncer.yml:

    custom_checks_dir: my_custom_checks\n\nmanifest_checks:\n    - name: check_model_deprecation_date\n      include: ^models/staging/legacy_erp\n
"},{"location":"checks/","title":"Checks","text":"

dbt-bouncer runs checks against artifacts from dbt. These checks fall into three categories:

  • Catalog checks:
    • Catalog Sources
    • Columns
  • Manifest checks:
    • Exposures
    • Lineage
    • Macros
    • Metadata
    • Models
    • Semantic Models
    • Sources
    • Unit Tests
  • Run Results checks:
    • Run Results
"},{"location":"checks/catalog/check_catalog_sources/","title":"Catalog Checks: Catalog Sources","text":"

Note

The below checks require both catalog.json and manifest.json to be present.

"},{"location":"checks/catalog/check_catalog_sources/#catalog.check_catalog_sources.CheckSourceColumnsAreAllDocumented","title":"CheckSourceColumnsAreAllDocumented","text":"

All columns in a source should be included in the source's properties file, i.e. .yml file.

Receives at execution time:

Name Type Description catalog_source CatalogTable

The CatalogTable object to check.

sources List[DbtBouncerSourceBase]

List of DbtBouncerSourceBase objects parsed from catalog.json.

Other Parameters (passed via config file):

Name Type Description exclude Optional[str]

Regex pattern to match the source path (i.e the .yml file where the source is configured). Source paths that match the pattern will not be checked.

include Optional[str]

Regex pattern to match the source path (i.e the .yml file where the source is configured). Only source paths that match the pattern will be checked.

severity Optional[Literal['error', 'warn']]

Severity level of the check. Default: error.

Example(s):

catalog_checks:\n    - name: check_source_columns_are_all_documented\n

Source code in src/dbt_bouncer/checks/catalog/check_catalog_sources.py
class CheckSourceColumnsAreAllDocumented(BaseCheck):\n    \"\"\"All columns in a source should be included in the source's properties file, i.e. `.yml` file.\n\n    Receives:\n        catalog_source (CatalogTable): The CatalogTable object to check.\n        sources (List[DbtBouncerSourceBase]): List of DbtBouncerSourceBase objects parsed from `catalog.json`.\n\n    Other Parameters:\n        exclude (Optional[str]): Regex pattern to match the source path (i.e the .yml file where the source is configured). Source paths that match the pattern will not be checked.\n        include (Optional[str]): Regex pattern to match the source path (i.e the .yml file where the source is configured). Only source paths that match the pattern will be checked.\n        severity (Optional[Literal[\"error\", \"warn\"]]): Severity level of the check. Default: `error`.\n\n    Example(s):\n        ```yaml\n        catalog_checks:\n            - name: check_source_columns_are_all_documented\n        ```\n\n    \"\"\"\n\n    catalog_source: \"CatalogTable\" = Field(default=None)\n    name: Literal[\"check_source_columns_are_all_documented\"]\n    sources: List[\"DbtBouncerSourceBase\"] = Field(default=[])\n\n    def execute(self) -> None:\n        \"\"\"Execute the check.\"\"\"\n        source = next(\n            s for s in self.sources if s.unique_id == self.catalog_source.unique_id\n        )\n        undocumented_columns = [\n            v.name\n            for _, v in self.catalog_source.columns.items()\n            if v.name not in source.columns\n        ]\n        assert not undocumented_columns, f\"`{self.catalog_source.unique_id}` has columns that are not included in the sources properties file: {undocumented_columns}\"\n
"},{"location":"checks/catalog/check_columns/","title":"Catalog Checks: Columns","text":"

Note

The below checks require both catalog.json and manifest.json to be present.

"},{"location":"checks/catalog/check_columns/#catalog.check_columns.CheckColumnDescriptionPopulated","title":"CheckColumnDescriptionPopulated","text":"

Columns must have a populated description.

Receives at execution time:

Name Type Description catalog_node CatalogTable

The CatalogTable object to check.

models List[DbtBouncerModelBase]

List of DbtBouncerModelBase objects parsed from manifest.json.

Other Parameters (passed via config file):

Name Type Description exclude Optional[str]

Regex pattern to match the model path. Model paths that match the pattern will not be checked.

include Optional[str]

Regex pattern to match the model path. Only model paths that match the pattern will be checked.

severity Optional[Literal['error', 'warn']]

Severity level of the check. Default: error.

Example(s):

manifest_checks:\n    - name: check_column_description_populated\n      include: ^models/marts\n

Source code in src/dbt_bouncer/checks/catalog/check_columns.py
class CheckColumnDescriptionPopulated(BaseCheck):\n    \"\"\"Columns must have a populated description.\n\n    Receives:\n        catalog_node (CatalogTable): The CatalogTable object to check.\n        models (List[DbtBouncerModelBase]): List of DbtBouncerModelBase objects parsed from `manifest.json`.\n\n    Other Parameters:\n        exclude (Optional[str]): Regex pattern to match the model path. Model paths that match the pattern will not be checked.\n        include (Optional[str]): Regex pattern to match the model path. Only model paths that match the pattern will be checked.\n        severity (Optional[Literal[\"error\", \"warn\"]]): Severity level of the check. Default: `error`.\n\n    Example(s):\n        ```yaml\n        manifest_checks:\n            - name: check_column_description_populated\n              include: ^models/marts\n        ```\n\n    \"\"\"\n\n    catalog_node: \"CatalogTable\" = Field(default=None)\n    models: List[\"DbtBouncerModelBase\"] = Field(default=[])\n    name: Literal[\"check_column_description_populated\"]\n\n    def execute(self) -> None:\n        \"\"\"Execute the check.\"\"\"\n        if self.catalog_node.unique_id.split(\".\")[0] == \"model\":\n            model = next(\n                m for m in self.models if m.unique_id == self.catalog_node.unique_id\n            )\n            non_complying_columns = []\n            for _, v in self.catalog_node.columns.items():\n                if (\n                    model.columns.get(v.name) is None\n                    or len(model.columns[v.name].description.strip()) <= 4\n                ):\n                    non_complying_columns.append(v.name)\n\n            assert not non_complying_columns, f\"`{self.catalog_node.unique_id.split('.')[-1]}` has columns that do not have a populated description: {non_complying_columns}\"\n
"},{"location":"checks/catalog/check_columns/#catalog.check_columns.CheckColumnNameCompliesToColumnType","title":"CheckColumnNameCompliesToColumnType","text":"

Columns with specified data types must comply to the specified regexp naming pattern.

Parameters:

Name Type Description Default column_name_pattern str

Regex pattern to match the model name.

required types List[str]

List of data types to check.

required

Receives at execution time:

Name Type Description catalog_node CatalogTable

The CatalogTable object to check.

Other Parameters (passed via config file):

Name Type Description exclude Optional[str]

Regex pattern to match the model path. Model paths that match the pattern will not be checked.

include Optional[str]

Regex pattern to match the model path. Only model paths that match the pattern will be checked.

severity Optional[Literal['error', 'warn']]

Severity level of the check. Default: error.

Example(s):

catalog_checks:\n    # DATE columns must end with \"_date\"\n    - name: check_column_name_complies_to_column_type\n      column_name_pattern: .*_date$\n      types:\n        - DATE\n
catalog_checks:\n    # BOOLEAN columns must start with \"is_\"\n    - name: check_column_name_complies_to_column_type\n      column_name_pattern: ^is_.*\n      types:\n        - BOOLEAN\n
catalog_checks:\n    # Columns of all types must consist of lowercase letters and underscores. Note that the specified types depend on the underlying database.\n    - name: check_column_name_complies_to_column_type\n      column_name_pattern: ^[a-z_]*$\n      types:\n        - BIGINT\n        - BOOLEAN\n        - DATE\n        - DOUBLE\n        - INTEGER\n        - VARCHAR\n

Source code in src/dbt_bouncer/checks/catalog/check_columns.py
class CheckColumnNameCompliesToColumnType(BaseCheck):\n    \"\"\"Columns with specified data types must comply to the specified regexp naming pattern.\n\n    Parameters:\n        column_name_pattern (str): Regex pattern to match the model name.\n        types (List[str]): List of data types to check.\n\n    Receives:\n        catalog_node (CatalogTable): The CatalogTable object to check.\n\n    Other Parameters:\n        exclude (Optional[str]): Regex pattern to match the model path. Model paths that match the pattern will not be checked.\n        include (Optional[str]): Regex pattern to match the model path. Only model paths that match the pattern will be checked.\n        severity (Optional[Literal[\"error\", \"warn\"]]): Severity level of the check. Default: `error`.\n\n    Example(s):\n        ```yaml\n        catalog_checks:\n            # DATE columns must end with \"_date\"\n            - name: check_column_name_complies_to_column_type\n              column_name_pattern: .*_date$\n              types:\n                - DATE\n        ```\n        ```yaml\n        catalog_checks:\n            # BOOLEAN columns must start with \"is_\"\n            - name: check_column_name_complies_to_column_type\n              column_name_pattern: ^is_.*\n              types:\n                - BOOLEAN\n        ```\n        ```yaml\n        catalog_checks:\n            # Columns of all types must consist of lowercase letters and underscores. Note that the specified types depend on the underlying database.\n            - name: check_column_name_complies_to_column_type\n              column_name_pattern: ^[a-z_]*$\n              types:\n                - BIGINT\n                - BOOLEAN\n                - DATE\n                - DOUBLE\n                - INTEGER\n                - VARCHAR\n        ```\n\n    \"\"\"\n\n    catalog_node: \"CatalogTable\" = Field(default=None)\n    column_name_pattern: str\n    name: Literal[\"check_column_name_complies_to_column_type\"]\n    types: List[str]\n\n    def execute(self) -> None:\n        \"\"\"Execute the check.\"\"\"\n        non_complying_columns = [\n            v.name\n            for _, v in self.catalog_node.columns.items()\n            if v.type in self.types\n            and re.compile(self.column_name_pattern.strip()).match(v.name) is None\n        ]\n\n        assert not non_complying_columns, f\"`{self.catalog_node.unique_id.split('.')[-1]}` has columns that don't comply with the specified regexp pattern (`{self.column_name_pattern}`): {non_complying_columns}\"\n
"},{"location":"checks/catalog/check_columns/#catalog.check_columns.CheckColumnsAreAllDocumented","title":"CheckColumnsAreAllDocumented","text":"

All columns in a model should be included in the model's properties file, i.e. .yml file.

Receives at execution time:

Name Type Description catalog_node CatalogTable

The CatalogTable object to check.

models List[DbtBouncerModel]

List of DbtBouncerModel objects parsed from manifest.json.

Other Parameters (passed via config file):

Name Type Description exclude Optional[str]

Regex pattern to match the model path. Model paths that match the pattern will not be checked.

include Optional[str]

Regex pattern to match the model path. Only model paths that match the pattern will be checked.

severity Optional[Literal['error', 'warn']]

Severity level of the check. Default: error.

Example(s):

catalog_checks:\n    - name: check_columns_are_all_documented\n

Source code in src/dbt_bouncer/checks/catalog/check_columns.py
class CheckColumnsAreAllDocumented(BaseCheck):\n    \"\"\"All columns in a model should be included in the model's properties file, i.e. `.yml` file.\n\n    Receives:\n        catalog_node (CatalogTable): The CatalogTable object to check.\n        models (List[DbtBouncerModel]): List of DbtBouncerModel objects parsed from `manifest.json`.\n\n    Other Parameters:\n        exclude (Optional[str]): Regex pattern to match the model path. Model paths that match the pattern will not be checked.\n        include (Optional[str]): Regex pattern to match the model path. Only model paths that match the pattern will be checked.\n        severity (Optional[Literal[\"error\", \"warn\"]]): Severity level of the check. Default: `error`.\n\n    Example(s):\n        ```yaml\n        catalog_checks:\n            - name: check_columns_are_all_documented\n        ```\n\n    \"\"\"\n\n    catalog_node: \"CatalogTable\" = Field(default=None)\n    models: List[\"DbtBouncerModelBase\"] = Field(default=[])\n    name: Literal[\"check_columns_are_all_documented\"]\n\n    def execute(self) -> None:\n        \"\"\"Execute the check.\"\"\"\n        if self.catalog_node.unique_id.split(\".\")[0] == \"model\":\n            model = next(\n                m for m in self.models if m.unique_id == self.catalog_node.unique_id\n            )\n            undocumented_columns = [\n                v.name\n                for _, v in self.catalog_node.columns.items()\n                if v.name not in model.columns\n            ]\n            assert not undocumented_columns, f\"`{self.catalog_node.unique_id.split('.')[-1]}` has columns that are not included in the models properties file: {undocumented_columns}\"\n
"},{"location":"checks/catalog/check_columns/#catalog.check_columns.CheckColumnsAreDocumentedInPublicModels","title":"CheckColumnsAreDocumentedInPublicModels","text":"

Columns should have a populated description in public models.

Receives at execution time:

Name Type Description catalog_node CatalogTable

The CatalogTable object to check.

models List[DbtBouncerModel]

List of DbtBouncerModel objects parsed from manifest.json.

Other Parameters (passed via config file):

Name Type Description exclude Optional[str]

Regex pattern to match the model path. Model paths that match the pattern will not be checked.

include Optional[str]

Regex pattern to match the model path. Only model paths that match the pattern will be checked.

severity Optional[Literal['error', 'warn']]

Severity level of the check. Default: error.

Example(s):

catalog_checks:\n    - name: check_columns_are_documented_in_public_models\n

Source code in src/dbt_bouncer/checks/catalog/check_columns.py
class CheckColumnsAreDocumentedInPublicModels(BaseCheck):\n    \"\"\"Columns should have a populated description in public models.\n\n    Receives:\n        catalog_node (CatalogTable): The CatalogTable object to check.\n        models (List[DbtBouncerModel]): List of DbtBouncerModel objects parsed from `manifest.json`.\n\n    Other Parameters:\n        exclude (Optional[str]): Regex pattern to match the model path. Model paths that match the pattern will not be checked.\n        include (Optional[str]): Regex pattern to match the model path. Only model paths that match the pattern will be checked.\n        severity (Optional[Literal[\"error\", \"warn\"]]): Severity level of the check. Default: `error`.\n\n    Example(s):\n        ```yaml\n        catalog_checks:\n            - name: check_columns_are_documented_in_public_models\n        ```\n\n    \"\"\"\n\n    catalog_node: \"CatalogTable\" = Field(default=None)\n    models: List[\"DbtBouncerModelBase\"] = Field(default=[])\n    name: Literal[\"check_columns_are_documented_in_public_models\"]\n\n    def execute(self) -> None:\n        \"\"\"Execute the check.\"\"\"\n        if self.catalog_node.unique_id.split(\".\")[0] == \"model\":\n            model = next(\n                m for m in self.models if m.unique_id == self.catalog_node.unique_id\n            )\n            non_complying_columns = []\n            for _, v in self.catalog_node.columns.items():\n                if model.access.value == \"public\":\n                    column_config = model.columns.get(v.name)\n                    if (\n                        column_config is None\n                        or len(column_config.description.strip()) < 4\n                    ):\n                        non_complying_columns.append(v.name)\n\n            assert not non_complying_columns, f\"`{self.catalog_node.unique_id.split('.')[-1]}` is a public model but has columns that don't have a populated description: {non_complying_columns}\"\n
"},{"location":"checks/catalog/check_columns/#catalog.check_columns.CheckColumnHasSpecifiedTest","title":"CheckColumnHasSpecifiedTest","text":"

Columns that match the specified regexp pattern must have a specified test.

Parameters:

Name Type Description Default column_name_pattern str

Regex pattern to match the column name.

required test_name str

Name of the test to check for.

required

Receives at execution time:

Name Type Description catalog_node CatalogTable

The CatalogTable object to check.

tests List[DbtBouncerTestBase]

List of DbtBouncerTestBase objects parsed from manifest.json.

Other Parameters (passed via config file):

Name Type Description exclude Optional[str]

Regex pattern to match the model path. Model paths that match the pattern will not be checked.

include Optional[str]

Regex pattern to match the model path. Only model paths that match the pattern will be checked.

severity Optional[Literal['error', 'warn']]

Severity level of the check. Default: error.

Example(s):

catalog_checks:\n    - name: check_column_has_specified_test\n      column_name_pattern: ^is_.*\n      test_name: not_null\n

Source code in src/dbt_bouncer/checks/catalog/check_columns.py
class CheckColumnHasSpecifiedTest(BaseCheck):\n    \"\"\"Columns that match the specified regexp pattern must have a specified test.\n\n    Parameters:\n        column_name_pattern (str): Regex pattern to match the column name.\n        test_name (str): Name of the test to check for.\n\n    Receives:\n        catalog_node (CatalogTable): The CatalogTable object to check.\n        tests (List[DbtBouncerTestBase]): List of DbtBouncerTestBase objects parsed from `manifest.json`.\n\n    Other Parameters:\n        exclude (Optional[str]): Regex pattern to match the model path. Model paths that match the pattern will not be checked.\n        include (Optional[str]): Regex pattern to match the model path. Only model paths that match the pattern will be checked.\n        severity (Optional[Literal[\"error\", \"warn\"]]): Severity level of the check. Default: `error`.\n\n    Example(s):\n        ```yaml\n        catalog_checks:\n            - name: check_column_has_specified_test\n              column_name_pattern: ^is_.*\n              test_name: not_null\n        ```\n\n    \"\"\"\n\n    catalog_node: \"CatalogTable\" = Field(default=None)\n    column_name_pattern: str\n    name: Literal[\"check_column_has_specified_test\"]\n    test_name: str\n    tests: List[\"DbtBouncerTestBase\"] = Field(default=[])\n\n    def execute(self) -> None:\n        \"\"\"Execute the check.\"\"\"\n        columns_to_check = [\n            v.name\n            for _, v in self.catalog_node.columns.items()\n            if re.compile(self.column_name_pattern.strip()).match(v.name) is not None\n        ]\n        relevant_tests = [\n            t\n            for t in self.tests\n            if hasattr(t, \"test_metadata\") is True\n            and hasattr(t, \"attached_node\") is True\n            and t.test_metadata.name == self.test_name\n            and t.attached_node == self.catalog_node.unique_id\n        ]\n        non_complying_columns = [\n            c\n            for c in columns_to_check\n            if f\"{self.catalog_node.unique_id}.{c}\"\n            not in [f\"{t.attached_node}.{t.column_name}\" for t in relevant_tests]\n        ]\n\n        assert not non_complying_columns, f\"`{self.catalog_node.unique_id.split('.')[-1]}` has columns that should have a `{self.test_name}` test: {non_complying_columns}\"\n
"},{"location":"checks/manifest/check_exposures/","title":"Manifest Checks: Exposures","text":"

Note

The below checks require manifest.json to be present.

"},{"location":"checks/manifest/check_exposures/#manifest.check_exposures.CheckExposureOnNonPublicModels","title":"CheckExposureOnNonPublicModels","text":"

Exposures should be based on public models only.

Receives at execution time:

Name Type Description exposure DbtBouncerExposureBase

The DbtBouncerExposureBase object to check.

models List[DbtBouncerModelBase]

List of DbtBouncerModelBase objects parsed from manifest.json.

Other Parameters (passed via config file):

Name Type Description exclude Optional[str]

Regex pattern to match the exposure path (i.e the .yml file where the exposure is configured). Exposure paths that match the pattern will not be checked.

include Optional[str]

Regex pattern to match the exposure path (i.e the .yml file where the exposure is configured). Only exposure paths that match the pattern will be checked.

severity Optional[Literal['error', 'warn']]

Severity level of the check. Default: error.

Example(s):

manifest_checks:\n    - name: check_exposure_based_on_non_public_models\n

Source code in src/dbt_bouncer/checks/manifest/check_exposures.py
class CheckExposureOnNonPublicModels(BaseCheck):\n    \"\"\"Exposures should be based on public models only.\n\n    Receives:\n        exposure (DbtBouncerExposureBase): The DbtBouncerExposureBase object to check.\n        models (List[DbtBouncerModelBase]): List of DbtBouncerModelBase objects parsed from `manifest.json`.\n\n    Other Parameters:\n        exclude (Optional[str]): Regex pattern to match the exposure path (i.e the .yml file where the exposure is configured). Exposure paths that match the pattern will not be checked.\n        include (Optional[str]): Regex pattern to match the exposure path (i.e the .yml file where the exposure is configured). Only exposure paths that match the pattern will be checked.\n        severity (Optional[Literal[\"error\", \"warn\"]]): Severity level of the check. Default: `error`.\n\n    Example(s):\n        ```yaml\n        manifest_checks:\n            - name: check_exposure_based_on_non_public_models\n        ```\n\n    \"\"\"\n\n    exposure: \"DbtBouncerExposureBase\" = Field(default=None)\n    models: List[\"DbtBouncerModelBase\"] = Field(default=[])\n    name: Literal[\"check_exposure_based_on_non_public_models\"]\n\n    def execute(self) -> None:\n        \"\"\"Execute the check.\"\"\"\n        non_public_upstream_dependencies = []\n        for model in self.exposure.depends_on.nodes:\n            if (\n                model.split(\".\")[0] == \"model\"\n                and model.split(\".\")[1] == self.exposure.package_name\n            ):\n                model = next(m for m in self.models if m.unique_id == model)\n                if model.access.value != \"public\":\n                    non_public_upstream_dependencies.append(model.name)\n\n        assert not non_public_upstream_dependencies, f\"`{self.exposure.name}` is based on a model(s) that is not public: {non_public_upstream_dependencies}.\"\n
"},{"location":"checks/manifest/check_exposures/#manifest.check_exposures.CheckExposureOnView","title":"CheckExposureOnView","text":"

Exposures should not be based on views.

Parameters:

Name Type Description Default materializations_to_include Optional[List[str]]

List of materializations to include in the check.

required

Receives at execution time:

Name Type Description exposure DbtBouncerExposureBase

The DbtBouncerExposureBase object to check.

models List[DbtBouncerModelBase]

List of DbtBouncerModelBase objects parsed from manifest.json.

Other Parameters (passed via config file):

Name Type Description exclude Optional[str]

Regex pattern to match the exposure path (i.e the .yml file where the exposure is configured). Exposure paths that match the pattern will not be checked.

include Optional[str]

Regex pattern to match the exposure path (i.e the .yml file where the exposure is configured). Only exposure paths that match the pattern will be checked.

severity Optional[Literal['error', 'warn']]

Severity level of the check. Default: error.

Example(s):

manifest_checks:\n    - name: check_exposure_based_on_view\n
manifest_checks:\n    - name: check_exposure_based_on_view\n      materializations_to_include:\n        - ephemeral\n        - my_custom_materialization\n        - view\n

Source code in src/dbt_bouncer/checks/manifest/check_exposures.py
class CheckExposureOnView(BaseCheck):\n    \"\"\"Exposures should not be based on views.\n\n    Parameters:\n        materializations_to_include (Optional[List[str]]): List of materializations to include in the check.\n\n    Receives:\n        exposure (DbtBouncerExposureBase): The DbtBouncerExposureBase object to check.\n        models (List[DbtBouncerModelBase]): List of DbtBouncerModelBase objects parsed from `manifest.json`.\n\n    Other Parameters:\n        exclude (Optional[str]): Regex pattern to match the exposure path (i.e the .yml file where the exposure is configured). Exposure paths that match the pattern will not be checked.\n        include (Optional[str]): Regex pattern to match the exposure path (i.e the .yml file where the exposure is configured). Only exposure paths that match the pattern will be checked.\n        severity (Optional[Literal[\"error\", \"warn\"]]): Severity level of the check. Default: `error`.\n\n    Example(s):\n        ```yaml\n        manifest_checks:\n            - name: check_exposure_based_on_view\n        ```\n        ```yaml\n        manifest_checks:\n            - name: check_exposure_based_on_view\n              materializations_to_include:\n                - ephemeral\n                - my_custom_materialization\n                - view\n        ```\n\n    \"\"\"\n\n    exposure: \"DbtBouncerExposureBase\" = Field(default=None)\n    materializations_to_include: List[str] = Field(\n        default=[\"ephemeral\", \"view\"],\n    )\n    models: List[\"DbtBouncerModelBase\"] = Field(default=[])\n    name: Literal[\"check_exposure_based_on_view\"]\n\n    def execute(self) -> None:\n        \"\"\"Execute the check.\"\"\"\n        non_table_upstream_dependencies = []\n        for model in self.exposure.depends_on.nodes:\n            if (\n                model.split(\".\")[0] == \"model\"\n                and model.split(\".\")[1] == self.exposure.package_name\n            ):\n                model = next(m for m in self.models if m.unique_id == model)\n                if model.config.materialized in self.materializations_to_include:\n                    non_table_upstream_dependencies.append(model.name)\n\n        assert not non_table_upstream_dependencies, f\"`{self.exposure.name}` is based on a model that is not a table: {non_table_upstream_dependencies}.\"\n
"},{"location":"checks/manifest/check_lineage/","title":"Manifest Checks: Lineage","text":"

Note

The below checks require manifest.json to be present.

"},{"location":"checks/manifest/check_lineage/#manifest.check_lineage.CheckLineagePermittedUpstreamModels","title":"CheckLineagePermittedUpstreamModels","text":"

Upstream models must have a path that matches the provided upstream_path_pattern.

Parameters:

Name Type Description Default upstream_path_pattern str

Regexp pattern to match the upstream model(s) path.

required

Receives at execution time:

Name Type Description manifest_obj DbtBouncerManifest

The manifest object.

model DbtBouncerModelBase

The DbtBouncerModelBase object to check.

models List[DbtBouncerModelBase]

List of DbtBouncerModelBase objects parsed from manifest.json.

Other Parameters (passed via config file):

Name Type Description exclude Optional[str]

Regex pattern to match the model path. Model paths that match the pattern will not be checked.

include Optional[str]

Regex pattern to match the model path. Only model paths that match the pattern will be checked.

severity Optional[Literal['error', 'warn']]

Severity level of the check. Default: error.

Example(s):

manifest_checks:\n    - name: check_lineage_permitted_upstream_models\n      include: ^models/staging\n      upstream_path_pattern: $^\n    - name: check_lineage_permitted_upstream_models\n      include: ^models/intermediate\n      upstream_path_pattern: ^models/staging|^models/intermediate\n    - name: check_lineage_permitted_upstream_models\n      include: ^models/marts\n      upstream_path_pattern: ^models/staging|^models/intermediate\n

Source code in src/dbt_bouncer/checks/manifest/check_lineage.py
class CheckLineagePermittedUpstreamModels(BaseCheck):\n    \"\"\"Upstream models must have a path that matches the provided `upstream_path_pattern`.\n\n    Parameters:\n        upstream_path_pattern (str): Regexp pattern to match the upstream model(s) path.\n\n    Receives:\n        manifest_obj (DbtBouncerManifest): The manifest object.\n        model (DbtBouncerModelBase): The DbtBouncerModelBase object to check.\n        models (List[DbtBouncerModelBase]): List of DbtBouncerModelBase objects parsed from `manifest.json`.\n\n    Other Parameters:\n        exclude (Optional[str]): Regex pattern to match the model path. Model paths that match the pattern will not be checked.\n        include (Optional[str]): Regex pattern to match the model path. Only model paths that match the pattern will be checked.\n        severity (Optional[Literal[\"error\", \"warn\"]]): Severity level of the check. Default: `error`.\n\n    Example(s):\n        ```yaml\n        manifest_checks:\n            - name: check_lineage_permitted_upstream_models\n              include: ^models/staging\n              upstream_path_pattern: $^\n            - name: check_lineage_permitted_upstream_models\n              include: ^models/intermediate\n              upstream_path_pattern: ^models/staging|^models/intermediate\n            - name: check_lineage_permitted_upstream_models\n              include: ^models/marts\n              upstream_path_pattern: ^models/staging|^models/intermediate\n        ```\n\n    \"\"\"\n\n    manifest_obj: \"DbtBouncerManifest\" = Field(default=None)\n    model: \"DbtBouncerModelBase\" = Field(default=None)\n    models: List[\"DbtBouncerModelBase\"] = Field(default=[])\n    name: Literal[\"check_lineage_permitted_upstream_models\"]\n    upstream_path_pattern: str\n\n    def execute(self) -> None:\n        \"\"\"Execute the check.\"\"\"\n        upstream_models = [\n            x\n            for x in self.model.depends_on.nodes\n            if x.split(\".\")[0] == \"model\"\n            and x.split(\".\")[1] == self.manifest_obj.manifest.metadata.project_name\n        ]\n        not_permitted_upstream_models = [\n            upstream_model\n            for upstream_model in upstream_models\n            if re.compile(self.upstream_path_pattern.strip()).match(\n                clean_path_str(\n                    next(\n                        m for m in self.models if m.unique_id == upstream_model\n                    ).original_file_path\n                ),\n            )\n            is None\n        ]\n        assert not not_permitted_upstream_models, f\"`{self.model.name}` references upstream models that are not permitted: {[m.split('.')[-1] for m in not_permitted_upstream_models]}.\"\n
"},{"location":"checks/manifest/check_lineage/#manifest.check_lineage.CheckLineageSeedCannotBeUsed","title":"CheckLineageSeedCannotBeUsed","text":"

Seed cannot be referenced in models with a path that matches the specified include config.

Receives at execution time:

Name Type Description model DbtBouncerModelBase

The DbtBouncerModelBase object to check.

Other Parameters (passed via config file):

Name Type Description exclude Optional[str]

Regex pattern to match the model path. Model paths that match the pattern will not be checked.

include Optional[str]

Regex pattern to match the model path. Only model paths that match the pattern will be checked.

severity Optional[Literal['error', 'warn']]

Severity level of the check. Default: error.

Example(s):

manifest_checks:\n    - name: check_lineage_seed_cannot_be_used\n      include: ^models/intermediate|^models/marts\n

Source code in src/dbt_bouncer/checks/manifest/check_lineage.py
class CheckLineageSeedCannotBeUsed(BaseCheck):\n    \"\"\"Seed cannot be referenced in models with a path that matches the specified `include` config.\n\n    Receives:\n        model (DbtBouncerModelBase): The DbtBouncerModelBase object to check.\n\n    Other Parameters:\n        exclude (Optional[str]): Regex pattern to match the model path. Model paths that match the pattern will not be checked.\n        include (Optional[str]): Regex pattern to match the model path. Only model paths that match the pattern will be checked.\n        severity (Optional[Literal[\"error\", \"warn\"]]): Severity level of the check. Default: `error`.\n\n    Example(s):\n        ```yaml\n        manifest_checks:\n            - name: check_lineage_seed_cannot_be_used\n              include: ^models/intermediate|^models/marts\n        ```\n\n    \"\"\"\n\n    model: \"DbtBouncerModelBase\" = Field(default=None)\n    name: Literal[\"check_lineage_seed_cannot_be_used\"]\n\n    def execute(self) -> None:\n        \"\"\"Execute the check.\"\"\"\n        assert not [\n            x for x in self.model.depends_on.nodes if x.split(\".\")[0] == \"seed\"\n        ], f\"`{self.model.name}` references a seed even though this is not permitted.\"\n
"},{"location":"checks/manifest/check_lineage/#manifest.check_lineage.CheckLineageSourceCannotBeUsed","title":"CheckLineageSourceCannotBeUsed","text":"

Sources cannot be referenced in models with a path that matches the specified include config.

Receives at execution time:

Name Type Description model DbtBouncerModelBase

The DbtBouncerModelBase object to check.

Other Parameters (passed via config file):

Name Type Description exclude Optional[str]

Regex pattern to match the model path. Model paths that match the pattern will not be checked.

include Optional[str]

Regex pattern to match the model path. Only model paths that match the pattern will be checked.

severity Optional[Literal['error', 'warn']]

Severity level of the check. Default: error.

Example(s):

manifest_checks:\n    - name: check_lineage_source_cannot_be_used\n      include: ^models/intermediate|^models/marts\n

Source code in src/dbt_bouncer/checks/manifest/check_lineage.py
class CheckLineageSourceCannotBeUsed(BaseCheck):\n    \"\"\"Sources cannot be referenced in models with a path that matches the specified `include` config.\n\n    Receives:\n        model (DbtBouncerModelBase): The DbtBouncerModelBase object to check.\n\n    Other Parameters:\n        exclude (Optional[str]): Regex pattern to match the model path. Model paths that match the pattern will not be checked.\n        include (Optional[str]): Regex pattern to match the model path. Only model paths that match the pattern will be checked.\n        severity (Optional[Literal[\"error\", \"warn\"]]): Severity level of the check. Default: `error`.\n\n    Example(s):\n        ```yaml\n        manifest_checks:\n            - name: check_lineage_source_cannot_be_used\n              include: ^models/intermediate|^models/marts\n        ```\n\n    \"\"\"\n\n    model: \"DbtBouncerModelBase\" = Field(default=None)\n    name: Literal[\"check_lineage_source_cannot_be_used\"]\n\n    def execute(self) -> None:\n        \"\"\"Execute the check.\"\"\"\n        assert not [\n            x for x in self.model.depends_on.nodes if x.split(\".\")[0] == \"source\"\n        ], f\"`{self.model.name}` references a source even though this is not permitted.\"\n
"},{"location":"checks/manifest/check_macros/","title":"Manifest Checks: Macros","text":"

Note

The below checks require manifest.json to be present.

"},{"location":"checks/manifest/check_macros/#manifest.check_macros.CheckMacroArgumentsDescriptionPopulated","title":"CheckMacroArgumentsDescriptionPopulated","text":"

Macro arguments must have a populated description.

Receives at execution time:

Name Type Description macro Macros

The Macros object to check.

Other Parameters (passed via config file):

Name Type Description exclude Optional[str]

Regex pattern to match the macro path. Macro paths that match the pattern will not be checked.

include Optional[str]

Regex pattern to match the macro path. Only macro paths that match the pattern will be checked.

severity Optional[Literal['error', 'warn']]

Severity level of the check. Default: error.

Example(s):

manifest_checks:\n    - name: check_macro_arguments_description_populated\n
# Only \"common\" macros need to have their arguments populated\nmanifest_checks:\n    - name: check_macro_arguments_description_populated\n      include: ^macros/common\n

Source code in src/dbt_bouncer/checks/manifest/check_macros.py
class CheckMacroArgumentsDescriptionPopulated(BaseCheck):\n    \"\"\"Macro arguments must have a populated description.\n\n    Receives:\n        macro (Macros): The Macros object to check.\n\n    Other Parameters:\n        exclude (Optional[str]): Regex pattern to match the macro path. Macro paths that match the pattern will not be checked.\n        include (Optional[str]): Regex pattern to match the macro path. Only macro paths that match the pattern will be checked.\n        severity (Optional[Literal[\"error\", \"warn\"]]): Severity level of the check. Default: `error`.\n\n    Example(s):\n        ```yaml\n        manifest_checks:\n            - name: check_macro_arguments_description_populated\n        ```\n        ```yaml\n        # Only \"common\" macros need to have their arguments populated\n        manifest_checks:\n            - name: check_macro_arguments_description_populated\n              include: ^macros/common\n        ```\n\n    \"\"\"\n\n    macro: \"Macros\" = Field(default=None)\n    name: Literal[\"check_macro_arguments_description_populated\"]\n\n    def execute(self) -> None:\n        \"\"\"Execute the check.\"\"\"\n        environment = jinja2.Environment(autoescape=True, extensions=[TagExtension])\n        ast = environment.parse(self.macro.macro_sql)\n\n        if hasattr(ast.body[0], \"args\"):\n            # Assume macro is a \"true\" macro\n            macro_arguments = [a.name for a in ast.body[0].args]\n        else:\n            if \"materialization\" in [\n                x.value.value\n                for x in ast.body[0].nodes[0].kwargs  # type: ignore[attr-defined]\n                if isinstance(x.value, jinja2.nodes.Const)\n            ]:\n                # Materializations don't have arguments\n                macro_arguments = []\n            else:\n                # Macro is a test\n                test_macro = next(\n                    x\n                    for x in ast.body\n                    if not isinstance(x.nodes[0], jinja2.nodes.Call)  # type: ignore[attr-defined]\n                )\n                macro_arguments = [\n                    x.name\n                    for x in test_macro.nodes  # type: ignore[attr-defined]\n                    if isinstance(x, jinja2.nodes.Name)\n                ]\n\n        # macro_arguments: List of args parsed from macro SQL\n        # macro.arguments: List of args manually added to the properties file\n\n        non_complying_args = []\n        for arg in macro_arguments:\n            macro_doc_raw = [x for x in self.macro.arguments if x.name == arg]\n            if macro_doc_raw == [] or (\n                arg not in [x.name for x in self.macro.arguments]\n                or len(macro_doc_raw[0].description.strip()) <= 4\n            ):\n                non_complying_args.append(arg)\n\n        assert (\n            non_complying_args == []\n        ), f\"Macro `{self.macro.name}` does not have a populated description for the following argument(s): {non_complying_args}.\"\n
"},{"location":"checks/manifest/check_macros/#manifest.check_macros.CheckMacroCodeDoesNotContainRegexpPattern","title":"CheckMacroCodeDoesNotContainRegexpPattern","text":"

The raw code for a macro must not match the specified regexp pattern.

Parameters:

Name Type Description Default regexp_pattern str

The regexp pattern that should not be matched by the macro code.

required

Receives at execution time:

Name Type Description macro Macros

The Macros object to check.

Other Parameters (passed via config file):

Name Type Description exclude Optional[str]

Regex pattern to match the macro path. Macro paths that match the pattern will not be checked.

include Optional[str]

Regex pattern to match the macro path. Only macro paths that match the pattern will be checked.

severity Optional[Literal['error', 'warn']]

Severity level of the check. Default: error.

Example(s):

manifest_checks:\n    # Prefer `coalesce` over `ifnull`: https://docs.sqlfluff.com/en/stable/rules.html#sqlfluff.rules.sphinx.Rule_CV02\n    - name: check_macro_code_does_not_contain_regexp_pattern\n      regexp_pattern: .*[i][f][n][u][l][l].*\n

Source code in src/dbt_bouncer/checks/manifest/check_macros.py
class CheckMacroCodeDoesNotContainRegexpPattern(BaseCheck):\n    \"\"\"The raw code for a macro must not match the specified regexp pattern.\n\n    Parameters:\n        regexp_pattern (str): The regexp pattern that should not be matched by the macro code.\n\n    Receives:\n        macro (Macros): The Macros object to check.\n\n    Other Parameters:\n        exclude (Optional[str]): Regex pattern to match the macro path. Macro paths that match the pattern will not be checked.\n        include (Optional[str]): Regex pattern to match the macro path. Only macro paths that match the pattern will be checked.\n        severity (Optional[Literal[\"error\", \"warn\"]]): Severity level of the check. Default: `error`.\n\n    Example(s):\n        ```yaml\n        manifest_checks:\n            # Prefer `coalesce` over `ifnull`: https://docs.sqlfluff.com/en/stable/rules.html#sqlfluff.rules.sphinx.Rule_CV02\n            - name: check_macro_code_does_not_contain_regexp_pattern\n              regexp_pattern: .*[i][f][n][u][l][l].*\n        ```\n\n    \"\"\"\n\n    macro: \"Macros\" = Field(default=None)\n    name: Literal[\"check_macro_code_does_not_contain_regexp_pattern\"]\n    regexp_pattern: str\n\n    def execute(self) -> None:\n        \"\"\"Execute the check.\"\"\"\n        assert (\n            re.compile(self.regexp_pattern.strip(), flags=re.DOTALL).match(\n                self.macro.macro_sql\n            )\n            is None\n        ), f\"Macro `{self.macro.name}` contains a banned string: `{self.regexp_pattern.strip()}`.\"\n
"},{"location":"checks/manifest/check_macros/#manifest.check_macros.CheckMacroDescriptionPopulated","title":"CheckMacroDescriptionPopulated","text":"

Macros must have a populated description.

Receives at execution time:

Name Type Description macro Macros

The Macros object to check.

Other Parameters (passed via config file):

Name Type Description exclude Optional[str]

Regex pattern to match the macro path. Macro paths that match the pattern will not be checked.

include Optional[str]

Regex pattern to match the macro path. Only macro paths that match the pattern will be checked.

severity Optional[Literal['error', 'warn']]

Severity level of the check. Default: error.

Example(s):

manifest_checks:\n    - name: check_macro_description_populated\n
# Only \"common\" macros need to have a populated description\nmanifest_checks:\n    - name: check_macro_description_populated\n      include: ^macros/common\n

Source code in src/dbt_bouncer/checks/manifest/check_macros.py
class CheckMacroDescriptionPopulated(BaseCheck):\n    \"\"\"Macros must have a populated description.\n\n    Receives:\n        macro (Macros): The Macros object to check.\n\n    Other Parameters:\n        exclude (Optional[str]): Regex pattern to match the macro path. Macro paths that match the pattern will not be checked.\n        include (Optional[str]): Regex pattern to match the macro path. Only macro paths that match the pattern will be checked.\n        severity (Optional[Literal[\"error\", \"warn\"]]): Severity level of the check. Default: `error`.\n\n    Example(s):\n        ```yaml\n        manifest_checks:\n            - name: check_macro_description_populated\n        ```\n        ```yaml\n        # Only \"common\" macros need to have a populated description\n        manifest_checks:\n            - name: check_macro_description_populated\n              include: ^macros/common\n        ```\n\n    \"\"\"\n\n    macro: \"Macros\" = Field(default=None)\n    name: Literal[\"check_macro_description_populated\"]\n\n    def execute(self) -> None:\n        \"\"\"Execute the check.\"\"\"\n        assert (\n            len(self.macro.description.strip()) > 4\n        ), f\"Macro `{self.macro.name}` does not have a populated description.\"\n
"},{"location":"checks/manifest/check_macros/#manifest.check_macros.CheckMacroMaxNumberOfLines","title":"CheckMacroMaxNumberOfLines","text":"

Macros may not have more than the specified number of lines.

Parameters:

Name Type Description Default max_number_of_lines int

The maximum number of permitted lines.

required

Receives at execution time:

Name Type Description macro Macros

The Macros object to check.

Other Parameters (passed via config file):

Name Type Description exclude Optional[str]

Regex pattern to match the macro path. Macro paths that match the pattern will not be checked.

include Optional[str]

Regex pattern to match the macro path. Only macro paths that match the pattern will be checked.

severity Optional[Literal['error', 'warn']]

Severity level of the check. Default: error.

Example(s):

manifest_checks:\n    - name: check_macro_max_number_of_lines\n
manifest_checks:\n    - name: check_macro_max_number_of_lines\n      max_number_of_lines: 100\n

Source code in src/dbt_bouncer/checks/manifest/check_macros.py
class CheckMacroMaxNumberOfLines(BaseCheck):\n    \"\"\"Macros may not have more than the specified number of lines.\n\n    Parameters:\n        max_number_of_lines (int): The maximum number of permitted lines.\n\n    Receives:\n        macro (Macros): The Macros object to check.\n\n    Other Parameters:\n        exclude (Optional[str]): Regex pattern to match the macro path. Macro paths that match the pattern will not be checked.\n        include (Optional[str]): Regex pattern to match the macro path. Only macro paths that match the pattern will be checked.\n        severity (Optional[Literal[\"error\", \"warn\"]]): Severity level of the check. Default: `error`.\n\n    Example(s):\n        ```yaml\n        manifest_checks:\n            - name: check_macro_max_number_of_lines\n        ```\n        ```yaml\n        manifest_checks:\n            - name: check_macro_max_number_of_lines\n              max_number_of_lines: 100\n        ```\n\n    \"\"\"\n\n    macro: \"Macros\" = Field(default=None)\n    name: Literal[\"check_macro_max_number_of_lines\"]\n    max_number_of_lines: int = Field(default=50)\n\n    def execute(self) -> None:\n        \"\"\"Execute the check.\"\"\"\n        actual_number_of_lines = self.macro.macro_sql.count(\"\\n\") + 1\n\n        assert (\n            actual_number_of_lines <= self.max_number_of_lines\n        ), f\"Macro `{self.macro.name}` has {actual_number_of_lines} lines, this is more than the maximum permitted number of lines ({self.max_number_of_lines}).\"\n
"},{"location":"checks/manifest/check_macros/#manifest.check_macros.CheckMacroNameMatchesFileName","title":"CheckMacroNameMatchesFileName","text":"

Macros names must be the same as the file they are contained in.

Generic tests are also macros, however to document these tests the \"name\" value must be preceded with \"test_\".

Receives at execution time:

Name Type Description macro Macros

The Macros object to check.

Other Parameters (passed via config file):

Name Type Description exclude Optional[str]

Regex pattern to match the macro path. Macro paths that match the pattern will not be checked.

include Optional[str]

Regex pattern to match the macro path. Only macro paths that match the pattern will be checked.

severity Optional[Literal['error', 'warn']]

Severity level of the check. Default: error.

Example(s):

manifest_checks:\n    - name: check_macro_name_matches_file_name\n

Source code in src/dbt_bouncer/checks/manifest/check_macros.py
class CheckMacroNameMatchesFileName(BaseCheck):\n    \"\"\"Macros names must be the same as the file they are contained in.\n\n    Generic tests are also macros, however to document these tests the \"name\" value must be preceded with \"test_\".\n\n    Receives:\n        macro (Macros): The Macros object to check.\n\n    Other Parameters:\n        exclude (Optional[str]): Regex pattern to match the macro path. Macro paths that match the pattern will not be checked.\n        include (Optional[str]): Regex pattern to match the macro path. Only macro paths that match the pattern will be checked.\n        severity (Optional[Literal[\"error\", \"warn\"]]): Severity level of the check. Default: `error`.\n\n    Example(s):\n        ```yaml\n        manifest_checks:\n            - name: check_macro_name_matches_file_name\n        ```\n\n    \"\"\"\n\n    macro: \"Macros\" = Field(default=None)\n    name: Literal[\"check_macro_name_matches_file_name\"]\n\n    def execute(self) -> None:\n        \"\"\"Execute the check.\"\"\"\n        if self.macro.name.startswith(\"test_\"):\n            assert (\n                self.macro.name[5:]\n                == clean_path_str(self.macro.original_file_path)\n                .split(\"/\")[-1]\n                .split(\".\")[0]\n            ), f\"Macro `{self.macro.unique_id}` is not in a file named `{self.macro.name[5:]}.sql`.\"\n        else:\n            assert (\n                self.macro.name\n                == clean_path_str(self.macro.original_file_path)\n                .split(\"/\")[-1]\n                .split(\".\")[0]\n            ), f\"Macro `{self.macro.name}` is not in a file of the same name.\"\n
"},{"location":"checks/manifest/check_macros/#manifest.check_macros.CheckMacroPropertyFileLocation","title":"CheckMacroPropertyFileLocation","text":"

Macro properties files must follow the guidance provided by dbt here.

Receives at execution time:

Name Type Description macro Macros

The Macros object to check.

Other Parameters (passed via config file):

Name Type Description exclude Optional[str]

Regex pattern to match the macro path. Macro paths that match the pattern will not be checked.

include Optional[str]

Regex pattern to match the macro path. Only macro paths that match the pattern will be checked.

severity Optional[Literal['error', 'warn']]

Severity level of the check. Default: error.

Example(s):

manifest_checks:\n    - name: check_macro_property_file_location\n

Source code in src/dbt_bouncer/checks/manifest/check_macros.py
class CheckMacroPropertyFileLocation(BaseCheck):\n    \"\"\"Macro properties files must follow the guidance provided by dbt [here](https://docs.getdbt.com/best-practices/how-we-structure/5-the-rest-of-the-project#how-we-use-the-other-folders).\n\n    Receives:\n        macro (Macros): The Macros object to check.\n\n    Other Parameters:\n        exclude (Optional[str]): Regex pattern to match the macro path. Macro paths that match the pattern will not be checked.\n        include (Optional[str]): Regex pattern to match the macro path. Only macro paths that match the pattern will be checked.\n        severity (Optional[Literal[\"error\", \"warn\"]]): Severity level of the check. Default: `error`.\n\n    Example(s):\n        ```yaml\n        manifest_checks:\n            - name: check_macro_property_file_location\n        ```\n\n    \"\"\"\n\n    macro: \"Macros\" = Field(default=None)\n    name: Literal[\"check_macro_property_file_location\"]\n\n    def execute(self) -> None:\n        \"\"\"Execute the check.\"\"\"\n        expected_substr = \"_\".join(\n            clean_path_str(self.macro.original_file_path)[6:].split(\"/\")[:-1]\n        )\n\n        assert (\n            clean_path_str(self.macro.patch_path) is not None\n        ), f\"Macro `{self.macro.name}` is not defined in a `.yml` properties file.\"\n        properties_yml_name = clean_path_str(self.macro.patch_path).split(\"/\")[-1]\n\n        if clean_path_str(self.macro.original_file_path).startswith(\n            \"tests/\",\n        ):  # Do not check generic tests (which are also macros)\n            pass\n        elif expected_substr == \"\":  # i.e. macro in ./macros\n            assert (\n                properties_yml_name == \"_macros.yml\"\n            ), f\"The properties file for `{self.macro.name}` (`{properties_yml_name}`) should be `_macros.yml`.\"\n        else:\n            assert properties_yml_name.startswith(\n                \"_\",\n            ), f\"The properties file for `{self.macro.name}` (`{properties_yml_name}`) does not start with an underscore.\"\n            assert (\n                expected_substr in properties_yml_name\n            ), f\"The properties file for `{self.macro.name}` (`{properties_yml_name}`) does not contain the expected substring (`{expected_substr}`).\"\n            assert properties_yml_name.endswith(\n                \"__macros.yml\",\n            ), f\"The properties file for `{self.macro.name.name}` (`{properties_yml_name}`) does not end with `__macros.yml`.\"\n
"},{"location":"checks/manifest/check_metadata/","title":"Manifest Checks: Metadata","text":"

Note

The below checks require manifest.json to be present.

"},{"location":"checks/manifest/check_metadata/#manifest.check_metadata.CheckProjectName","title":"CheckProjectName","text":"

Enforce that the name of the dbt project matches a supplied regex. Generally used to enforce that project names conform to something like company_<DOMAIN>.

Parameters:

Name Type Description Default project_name_pattern str

Regex pattern to match the project name.

required

Receives at execution time:

Name Type Description manifest_obj DbtBouncerManifest

The manifest object.

Other Parameters (passed via config file):

Name Type Description severity Optional[Literal['error', 'warn']]

Severity level of the check. Default: error.

Example(s):

manifest_checks:\n    - name: check_project_name\n      project_name_pattern: ^awesome_company_\n

Source code in src/dbt_bouncer/checks/manifest/check_metadata.py
class CheckProjectName(BaseModel):\n    \"\"\"Enforce that the name of the dbt project matches a supplied regex. Generally used to enforce that project names conform to something like  `company_<DOMAIN>`.\n\n    Parameters:\n        project_name_pattern (str): Regex pattern to match the project name.\n\n    Receives:\n        manifest_obj (DbtBouncerManifest): The manifest object.\n\n    Other Parameters:\n        severity (Optional[Literal[\"error\", \"warn\"]]): Severity level of the check. Default: `error`.\n\n    Example(s):\n        ```yaml\n        manifest_checks:\n            - name: check_project_name\n              project_name_pattern: ^awesome_company_\n        ```\n\n    \"\"\"\n\n    model_config = ConfigDict(extra=\"forbid\")\n\n    index: Optional[int] = Field(\n        default=None,\n        description=\"Index to uniquely identify the check, calculated at runtime.\",\n    )\n    manifest_obj: \"DbtBouncerManifest\" = Field(default=None)\n    name: Literal[\"check_project_name\"]\n    project_name_pattern: str\n    severity: Optional[Literal[\"error\", \"warn\"]] = Field(\n        default=\"error\",\n        description=\"Severity of the check, one of 'error' or 'warn'.\",\n    )\n\n    def execute(self) -> None:\n        \"\"\"Execute the check.\"\"\"\n        assert (\n            re.compile(self.project_name_pattern.strip()).match(\n                self.manifest_obj.manifest.metadata.project_name,\n            )\n            is not None\n        ), f\"Project name (`{self.manifest_obj.manifest.metadata.project_name}`) does not conform to the supplied regex `({self.project_name_pattern.strip()})`.\"\n
"},{"location":"checks/manifest/check_models/","title":"Manifest Checks: Models","text":"

Note

The below checks require manifest.json to be present.

"},{"location":"checks/manifest/check_models/#manifest.check_models.CheckModelAccess","title":"CheckModelAccess","text":"

Models must have the specified access attribute. Requires dbt 1.7+.

Parameters:

Name Type Description Default access Literal['private', 'protected', 'public']

The access level to check for.

required

Receives at execution time:

Name Type Description model DbtBouncerModelBase

The DbtBouncerModelBase object to check.

Other Parameters (passed via config file):

Name Type Description exclude Optional[str]

Regex pattern to match the model path. Model paths that match the pattern will not be checked.

include Optional[str]

Regex pattern to match the model path. Only model paths that match the pattern will be checked.

severity Optional[Literal['error', 'warn']]

Severity level of the check. Default: error.

Example(s):

manifest_checks:\n    # Align with dbt best practices that marts should be `public`, everything else should be `protected`\n    - name: check_model_access\n      access: protected\n      include: ^models/intermediate\n    - name: check_model_access\n      access: public\n      include: ^models/marts\n    - name: check_model_access\n      access: protected\n      include: ^models/staging\n

Source code in src/dbt_bouncer/checks/manifest/check_models.py
class CheckModelAccess(BaseCheck):\n    \"\"\"Models must have the specified access attribute. Requires dbt 1.7+.\n\n    Parameters:\n        access (Literal[\"private\", \"protected\", \"public\"]): The access level to check for.\n\n    Receives:\n        model (DbtBouncerModelBase): The DbtBouncerModelBase object to check.\n\n    Other Parameters:\n        exclude (Optional[str]): Regex pattern to match the model path. Model paths that match the pattern will not be checked.\n        include (Optional[str]): Regex pattern to match the model path. Only model paths that match the pattern will be checked.\n        severity (Optional[Literal[\"error\", \"warn\"]]): Severity level of the check. Default: `error`.\n\n    Example(s):\n        ```yaml\n        manifest_checks:\n            # Align with dbt best practices that marts should be `public`, everything else should be `protected`\n            - name: check_model_access\n              access: protected\n              include: ^models/intermediate\n            - name: check_model_access\n              access: public\n              include: ^models/marts\n            - name: check_model_access\n              access: protected\n              include: ^models/staging\n        ```\n\n    \"\"\"\n\n    access: Literal[\"private\", \"protected\", \"public\"]\n    model: \"DbtBouncerModelBase\" = Field(default=None)\n    name: Literal[\"check_model_access\"]\n\n    def execute(self) -> None:\n        \"\"\"Execute the check.\"\"\"\n        assert (\n            self.model.access.value == self.access\n        ), f\"`{self.model.name}` has `{self.model.access.value}` access, it should have access `{self.access}`.\"\n
"},{"location":"checks/manifest/check_models/#manifest.check_models.CheckModelContractsEnforcedForPublicModel","title":"CheckModelContractsEnforcedForPublicModel","text":"

Public models must have contracts enforced.

Receives at execution time:

Name Type Description model DbtBouncerModelBase

The DbtBouncerModelBase object to check.

Other Parameters (passed via config file):

Name Type Description exclude Optional[str]

Regex pattern to match the model path. Model paths that match the pattern will not be checked.

include Optional[str]

Regex pattern to match the model path. Only model paths that match the pattern will be checked.

severity Optional[Literal['error', 'warn']]

Severity level of the check. Default: error.

Example(s):

manifest_checks:\n    - name: check_model_contract_enforced_for_public_model\n

Source code in src/dbt_bouncer/checks/manifest/check_models.py
class CheckModelContractsEnforcedForPublicModel(BaseCheck):\n    \"\"\"Public models must have contracts enforced.\n\n    Receives:\n        model (DbtBouncerModelBase): The DbtBouncerModelBase object to check.\n\n    Other Parameters:\n        exclude (Optional[str]): Regex pattern to match the model path. Model paths that match the pattern will not be checked.\n        include (Optional[str]): Regex pattern to match the model path. Only model paths that match the pattern will be checked.\n        severity (Optional[Literal[\"error\", \"warn\"]]): Severity level of the check. Default: `error`.\n\n    Example(s):\n        ```yaml\n        manifest_checks:\n            - name: check_model_contract_enforced_for_public_model\n        ```\n\n    \"\"\"\n\n    model: \"DbtBouncerModelBase\" = Field(default=None)\n    name: Literal[\"check_model_contract_enforced_for_public_model\"]\n\n    def execute(self) -> None:\n        \"\"\"Execute the check.\"\"\"\n        if self.model.access.value == \"public\":\n            assert (\n                self.model.contract.enforced is True\n            ), f\"`{self.model.name}` is a public model but does not have contracts enforced.\"\n
"},{"location":"checks/manifest/check_models/#manifest.check_models.CheckModelDescriptionPopulated","title":"CheckModelDescriptionPopulated","text":"

Models must have a populated description.

Receives at execution time:

Name Type Description model DbtBouncerModelBase

The DbtBouncerModelBase object to check.

Other Parameters (passed via config file):

Name Type Description exclude Optional[str]

Regex pattern to match the model path. Model paths that match the pattern will not be checked.

include Optional[str]

Regex pattern to match the model path. Only model paths that match the pattern will be checked.

severity Optional[Literal['error', 'warn']]

Severity level of the check. Default: error.

Example(s):

manifest_checks:\n    - name: check_model_description_populated\n

Source code in src/dbt_bouncer/checks/manifest/check_models.py
class CheckModelDescriptionPopulated(BaseCheck):\n    \"\"\"Models must have a populated description.\n\n    Receives:\n        model (DbtBouncerModelBase): The DbtBouncerModelBase object to check.\n\n    Other Parameters:\n        exclude (Optional[str]): Regex pattern to match the model path. Model paths that match the pattern will not be checked.\n        include (Optional[str]): Regex pattern to match the model path. Only model paths that match the pattern will be checked.\n        severity (Optional[Literal[\"error\", \"warn\"]]): Severity level of the check. Default: `error`.\n\n    Example(s):\n        ```yaml\n        manifest_checks:\n            - name: check_model_description_populated\n        ```\n\n    \"\"\"\n\n    model: \"DbtBouncerModelBase\" = Field(default=None)\n    name: Literal[\"check_model_description_populated\"]\n\n    def execute(self) -> None:\n        \"\"\"Execute the check.\"\"\"\n        assert (\n            len(self.model.description.strip()) > 4\n        ), f\"`{self.model.name}` does not have a populated description.\"\n
"},{"location":"checks/manifest/check_models/#manifest.check_models.CheckModelsDocumentationCoverage","title":"CheckModelsDocumentationCoverage","text":"

Set the minimum percentage of models that have a populated description.

Parameters:

Name Type Description Default min_model_documentation_coverage_pct float

The minimum percentage of models that must have a populated description.

required

Receives at execution time:

Name Type Description models List[DbtBouncerModelBase]

List of DbtBouncerModelBase objects parsed from manifest.json.

Other Parameters (passed via config file):

Name Type Description severity Optional[Literal['error', 'warn']]

Severity level of the check. Default: error.

Example(s):

manifest_checks:\n    - name: check_model_description_populated\n      min_model_documentation_coverage_pct: 90\n

Source code in src/dbt_bouncer/checks/manifest/check_models.py
class CheckModelsDocumentationCoverage(BaseModel):\n    \"\"\"Set the minimum percentage of models that have a populated description.\n\n    Parameters:\n        min_model_documentation_coverage_pct (float): The minimum percentage of models that must have a populated description.\n\n    Receives:\n        models (List[DbtBouncerModelBase]): List of DbtBouncerModelBase objects parsed from `manifest.json`.\n\n    Other Parameters:\n        severity (Optional[Literal[\"error\", \"warn\"]]): Severity level of the check. Default: `error`.\n\n    Example(s):\n        ```yaml\n        manifest_checks:\n            - name: check_model_description_populated\n              min_model_documentation_coverage_pct: 90\n        ```\n\n    \"\"\"\n\n    model_config = ConfigDict(extra=\"forbid\")\n\n    index: Optional[int] = Field(\n        default=None,\n        description=\"Index to uniquely identify the check, calculated at runtime.\",\n    )\n    min_model_documentation_coverage_pct: int = Field(\n        default=100,\n        ge=0,\n        le=100,\n    )\n    models: List[\"DbtBouncerModelBase\"] = Field(default=[])\n    name: Literal[\"check_model_documentation_coverage\"]\n    severity: Optional[Literal[\"error\", \"warn\"]] = Field(\n        default=\"error\",\n        description=\"Severity of the check, one of 'error' or 'warn'.\",\n    )\n\n    def execute(self) -> None:\n        \"\"\"Execute the check.\"\"\"\n        num_models = len(self.models)\n        models_with_description = []\n        for model in self.models:\n            if len(model.description.strip()) > 4:\n                models_with_description.append(model.unique_id)\n\n        num_models_with_descriptions = len(models_with_description)\n        model_description_coverage_pct = (\n            num_models_with_descriptions / num_models\n        ) * 100\n\n        assert (\n            model_description_coverage_pct >= self.min_model_documentation_coverage_pct\n        ), f\"Only {model_description_coverage_pct}% of models have a populated description, this is less than the permitted minimum of {self.min_model_documentation_coverage_pct}%.\"\n
"},{"location":"checks/manifest/check_models/#manifest.check_models.CheckModelDocumentedInSameDirectory","title":"CheckModelDocumentedInSameDirectory","text":"

Models must be documented in the same directory where they are defined (i.e. .yml and .sql files are in the same directory).

Receives at execution time:

Name Type Description model DbtBouncerModelBase

The DbtBouncerModelBase object to check.

Other Parameters (passed via config file):

Name Type Description exclude Optional[str]

Regex pattern to match the model path. Model paths that match the pattern will not be checked.

include Optional[str]

Regex pattern to match the model path. Only model paths that match the pattern will be checked.

severity Optional[Literal['error', 'warn']]

Severity level of the check. Default: error.

Example(s):

manifest_checks:\n    - name: check_model_documented_in_same_directory\n

Source code in src/dbt_bouncer/checks/manifest/check_models.py
class CheckModelDocumentedInSameDirectory(BaseCheck):\n    \"\"\"Models must be documented in the same directory where they are defined (i.e. `.yml` and `.sql` files are in the same directory).\n\n    Receives:\n        model (DbtBouncerModelBase): The DbtBouncerModelBase object to check.\n\n    Other Parameters:\n        exclude (Optional[str]): Regex pattern to match the model path. Model paths that match the pattern will not be checked.\n        include (Optional[str]): Regex pattern to match the model path. Only model paths that match the pattern will be checked.\n        severity (Optional[Literal[\"error\", \"warn\"]]): Severity level of the check. Default: `error`.\n\n    Example(s):\n        ```yaml\n        manifest_checks:\n            - name: check_model_documented_in_same_directory\n        ```\n\n    \"\"\"\n\n    model: \"DbtBouncerModelBase\" = Field(default=None)\n    name: Literal[\"check_model_documented_in_same_directory\"]\n\n    def execute(self) -> None:\n        \"\"\"Execute the check.\"\"\"\n        model_sql_dir = clean_path_str(self.model.original_file_path).split(\"/\")[:-1]\n        assert (  # noqa: PT018\n            hasattr(self.model, \"patch_path\")\n            and clean_path_str(self.model.patch_path) is not None\n        ), f\"`{self.model.name}` is not documented.\"\n\n        model_doc_dir = clean_path_str(\n            self.model.patch_path[\n                clean_path_str(self.model.patch_path).find(\"models\") :\n            ]\n        ).split(\"/\")[:-1]\n\n        assert (\n            model_doc_dir == model_sql_dir\n        ), f\"`{self.model.name}` is documented in a different directory to the `.sql` file: `{'/'.join(model_doc_dir)}` vs `{'/'.join(model_sql_dir)}`.\"\n
"},{"location":"checks/manifest/check_models/#manifest.check_models.CheckModelCodeDoesNotContainRegexpPattern","title":"CheckModelCodeDoesNotContainRegexpPattern","text":"

The raw code for a model must not match the specified regexp pattern.

Parameters:

Name Type Description Default regexp_pattern str

The regexp pattern that should not be matched by the model code.

required

Receives at execution time:

Name Type Description model DbtBouncerModelBase

The DbtBouncerModelBase object to check.

Other Parameters (passed via config file):

Name Type Description exclude Optional[str]

Regex pattern to match the model path. Model paths that match the pattern will not be checked.

include Optional[str]

Regex pattern to match the model path. Only model paths that match the pattern will be checked.

severity Optional[Literal['error', 'warn']]

Severity level of the check. Default: error.

Example(s):

manifest_checks:\n    # Prefer `coalesce` over `ifnull`: https://docs.sqlfluff.com/en/stable/rules.html#sqlfluff.rules.sphinx.Rule_CV02\n    - name: check_model_code_does_not_contain_regexp_pattern\n      regexp_pattern: .*[i][f][n][u][l][l].*\n

Source code in src/dbt_bouncer/checks/manifest/check_models.py
class CheckModelCodeDoesNotContainRegexpPattern(BaseCheck):\n    \"\"\"The raw code for a model must not match the specified regexp pattern.\n\n    Parameters:\n        regexp_pattern (str): The regexp pattern that should not be matched by the model code.\n\n    Receives:\n        model (DbtBouncerModelBase): The DbtBouncerModelBase object to check.\n\n    Other Parameters:\n        exclude (Optional[str]): Regex pattern to match the model path. Model paths that match the pattern will not be checked.\n        include (Optional[str]): Regex pattern to match the model path. Only model paths that match the pattern will be checked.\n        severity (Optional[Literal[\"error\", \"warn\"]]): Severity level of the check. Default: `error`.\n\n    Example(s):\n        ```yaml\n        manifest_checks:\n            # Prefer `coalesce` over `ifnull`: https://docs.sqlfluff.com/en/stable/rules.html#sqlfluff.rules.sphinx.Rule_CV02\n            - name: check_model_code_does_not_contain_regexp_pattern\n              regexp_pattern: .*[i][f][n][u][l][l].*\n        ```\n\n    \"\"\"\n\n    model: \"DbtBouncerModelBase\" = Field(default=None)\n    name: Literal[\"check_model_code_does_not_contain_regexp_pattern\"]\n    regexp_pattern: str\n\n    def execute(self) -> None:\n        \"\"\"Execute the check.\"\"\"\n        assert (\n            re.compile(self.regexp_pattern.strip(), flags=re.DOTALL).match(\n                self.model.raw_code\n            )\n            is None\n        ), f\"`{self.model.name}` contains a banned string: `{self.regexp_pattern.strip()}`.\"\n
"},{"location":"checks/manifest/check_models/#manifest.check_models.CheckModelDependsOnMultipleSources","title":"CheckModelDependsOnMultipleSources","text":"

Models cannot reference more than one source.

Parameters:

Name Type Description Default model DbtBouncerModelBase

The DbtBouncerModelBase object to check.

required

Other Parameters (passed via config file):

Name Type Description exclude Optional[str]

Regex pattern to match the model path. Model paths that match the pattern will not be checked.

include Optional[str]

Regex pattern to match the model path. Only model paths that match the pattern will be checked.

severity Optional[Literal['error', 'warn']]

Severity level of the check. Default: error.

Example(s):

manifest_checks:\n    - name: check_model_depends_on_multiple_sources\n

Source code in src/dbt_bouncer/checks/manifest/check_models.py
class CheckModelDependsOnMultipleSources(BaseCheck):\n    \"\"\"Models cannot reference more than one source.\n\n    Parameters:\n        model (DbtBouncerModelBase): The DbtBouncerModelBase object to check.\n\n    Other Parameters:\n        exclude (Optional[str]): Regex pattern to match the model path. Model paths that match the pattern will not be checked.\n        include (Optional[str]): Regex pattern to match the model path. Only model paths that match the pattern will be checked.\n        severity (Optional[Literal[\"error\", \"warn\"]]): Severity level of the check. Default: `error`.\n\n    Example(s):\n        ```yaml\n        manifest_checks:\n            - name: check_model_depends_on_multiple_sources\n        ```\n\n    \"\"\"\n\n    model: \"DbtBouncerModelBase\" = Field(default=None)\n    name: Literal[\"check_model_depends_on_multiple_sources\"]\n\n    def execute(self) -> None:\n        \"\"\"Execute the check.\"\"\"\n        num_reffed_sources = sum(\n            x.split(\".\")[0] == \"source\" for x in self.model.depends_on.nodes\n        )\n        assert (\n            num_reffed_sources <= 1\n        ), f\"`{self.model.name}` references more than one source.\"\n
"},{"location":"checks/manifest/check_models/#manifest.check_models.CheckModelDirectories","title":"CheckModelDirectories","text":"

Only specified sub-directories are permitted.

Parameters:

Name Type Description Default include str

Regex pattern to the directory to check.

required permitted_sub_directories List[str]

List of permitted sub-directories.

required

Receives at execution time:

Name Type Description model DbtBouncerModelBase

The DbtBouncerModelBase object to check.

Example(s):

manifest_checks:\n- name: check_model_directories\n  include: models\n  permitted_sub_directories:\n    - intermediate\n    - marts\n    - staging\n
# Restrict sub-directories within `./models/staging`\n- name: check_model_directories\n  include: ^models/staging\n  permitted_sub_directories:\n    - crm\n    - payments\n

Source code in src/dbt_bouncer/checks/manifest/check_models.py
class CheckModelDirectories(BaseCheck):\n    \"\"\"Only specified sub-directories are permitted.\n\n    Parameters:\n        include (str): Regex pattern to the directory to check.\n        permitted_sub_directories (List[str]): List of permitted sub-directories.\n\n    Receives:\n        model (DbtBouncerModelBase): The DbtBouncerModelBase object to check.\n\n    Example(s):\n        ```yaml\n        manifest_checks:\n        - name: check_model_directories\n          include: models\n          permitted_sub_directories:\n            - intermediate\n            - marts\n            - staging\n        ```\n        ```yaml\n        # Restrict sub-directories within `./models/staging`\n        - name: check_model_directories\n          include: ^models/staging\n          permitted_sub_directories:\n            - crm\n            - payments\n        ```\n\n    \"\"\"\n\n    include: str\n    model: \"DbtBouncerModelBase\" = Field(default=None)\n    name: Literal[\"check_model_directories\"]\n    permitted_sub_directories: List[str]\n\n    def execute(self) -> None:\n        \"\"\"Execute the check.\"\"\"\n        matched_path = re.compile(self.include.strip()).match(\n            clean_path_str(self.model.original_file_path)\n        )\n        path_after_match = clean_path_str(self.model.original_file_path)[\n            matched_path.end() + 1 :\n        ]\n\n        assert (\n            path_after_match.split(\"/\")[0] in self.permitted_sub_directories\n        ), f\"`{self.model.name}` is located in `{self.model.original_file_path.split('/')[1]}`, this is not a valid sub-directory.\"\n
"},{"location":"checks/manifest/check_models/#manifest.check_models.CheckModelHasContractsEnforced","title":"CheckModelHasContractsEnforced","text":"

Model must have contracts enforced.

Receives at execution time:

Name Type Description model DbtBouncerModelBase

The DbtBouncerModelBase object to check.

Other Parameters (passed via config file):

Name Type Description exclude Optional[str]

Regex pattern to match the model path. Model paths that match the pattern will not be checked.

include Optional[str]

Regex pattern to match the model path. Only model paths that match the pattern will be checked.

severity Optional[Literal['error', 'warn']]

Severity level of the check. Default: error.

Example(s):

manifest_checks:\n    - name: check_model_has_contracts_enforced\n      include: ^models/marts\n

Source code in src/dbt_bouncer/checks/manifest/check_models.py
class CheckModelHasContractsEnforced(BaseCheck):\n    \"\"\"Model must have contracts enforced.\n\n    Receives:\n        model (DbtBouncerModelBase): The DbtBouncerModelBase object to check.\n\n    Other Parameters:\n        exclude (Optional[str]): Regex pattern to match the model path. Model paths that match the pattern will not be checked.\n        include (Optional[str]): Regex pattern to match the model path. Only model paths that match the pattern will be checked.\n        severity (Optional[Literal[\"error\", \"warn\"]]): Severity level of the check. Default: `error`.\n\n    Example(s):\n        ```yaml\n        manifest_checks:\n            - name: check_model_has_contracts_enforced\n              include: ^models/marts\n        ```\n\n    \"\"\"\n\n    model: \"DbtBouncerModelBase\" = Field(default=None)\n    name: Literal[\"check_model_has_contracts_enforced\"]\n\n    def execute(self) -> None:\n        \"\"\"Execute the check.\"\"\"\n        assert (\n            self.model.contract.enforced is True\n        ), f\"`{self.model.name}` does not have contracts enforced.\"\n
"},{"location":"checks/manifest/check_models/#manifest.check_models.CheckModelHasMetaKeys","title":"CheckModelHasMetaKeys","text":"

The meta config for models must have the specified keys.

Parameters:

Name Type Description Default keys NestedDict

A list (that may contain sub-lists) of required keys.

required model DbtBouncerModelBase

The DbtBouncerModelBase object to check.

required

Other Parameters (passed via config file):

Name Type Description exclude Optional[str]

Regex pattern to match the model path. Model paths that match the pattern will not be checked.

include Optional[str]

Regex pattern to match the model path. Only model paths that match the pattern will be checked.

severity Optional[Literal['error', 'warn']]

Severity level of the check. Default: error.

Example(s):

manifest_checks:\n    - name: check_model_has_meta_keys\n      keys:\n        - maturity\n        - owner\n

Source code in src/dbt_bouncer/checks/manifest/check_models.py
class CheckModelHasMetaKeys(BaseCheck):\n    \"\"\"The `meta` config for models must have the specified keys.\n\n    Parameters:\n        keys (NestedDict): A list (that may contain sub-lists) of required keys.\n        model (DbtBouncerModelBase): The DbtBouncerModelBase object to check.\n\n    Other Parameters:\n        exclude (Optional[str]): Regex pattern to match the model path. Model paths that match the pattern will not be checked.\n        include (Optional[str]): Regex pattern to match the model path. Only model paths that match the pattern will be checked.\n        severity (Optional[Literal[\"error\", \"warn\"]]): Severity level of the check. Default: `error`.\n\n    Example(s):\n        ```yaml\n        manifest_checks:\n            - name: check_model_has_meta_keys\n              keys:\n                - maturity\n                - owner\n        ```\n\n    \"\"\"\n\n    keys: NestedDict\n    model: \"DbtBouncerModelBase\" = Field(default=None)\n    name: Literal[\"check_model_has_meta_keys\"]\n\n    def execute(self) -> None:\n        \"\"\"Execute the check.\"\"\"\n        missing_keys = find_missing_meta_keys(\n            meta_config=self.model.meta,\n            required_keys=self.keys.model_dump(),\n        )\n        assert (\n            missing_keys == []\n        ), f\"`{self.model.name}` is missing the following keys from the `meta` config: {[x.replace('>>', '') for x in missing_keys]}\"\n
"},{"location":"checks/manifest/check_models/#manifest.check_models.CheckModelHasNoUpstreamDependencies","title":"CheckModelHasNoUpstreamDependencies","text":"

Identify if models have no upstream dependencies as this likely indicates hard-coded tables references.

Receives at execution time:

Name Type Description model DbtBouncerModelBase

The DbtBouncerModelBase object to check.

Other Parameters (passed via config file):

Name Type Description exclude Optional[str]

Regex pattern to match the model path. Model paths that match the pattern will not be checked.

include Optional[str]

Regex pattern to match the model path. Only model paths that match the pattern will be checked.

severity Optional[Literal['error', 'warn']]

Severity level of the check. Default: error.

Example(s):

manifest_checks:\n    - name: check_model_has_no_upstream_dependencies\n

Source code in src/dbt_bouncer/checks/manifest/check_models.py
class CheckModelHasNoUpstreamDependencies(BaseCheck):\n    \"\"\"Identify if models have no upstream dependencies as this likely indicates hard-coded tables references.\n\n    Receives:\n        model (DbtBouncerModelBase): The DbtBouncerModelBase object to check.\n\n    Other Parameters:\n        exclude (Optional[str]): Regex pattern to match the model path. Model paths that match the pattern will not be checked.\n        include (Optional[str]): Regex pattern to match the model path. Only model paths that match the pattern will be checked.\n        severity (Optional[Literal[\"error\", \"warn\"]]): Severity level of the check. Default: `error`.\n\n    Example(s):\n        ```yaml\n        manifest_checks:\n            - name: check_model_has_no_upstream_dependencies\n        ```\n\n    \"\"\"\n\n    model: \"DbtBouncerModelBase\" = Field(default=None)\n    name: Literal[\"check_model_has_no_upstream_dependencies\"]\n\n    def execute(self) -> None:\n        \"\"\"Execute the check.\"\"\"\n        assert (\n            len(self.model.depends_on.nodes) > 0\n        ), f\"`{self.model.name}` has no upstream dependencies, this likely indicates hard-coded tables references.\"\n
"},{"location":"checks/manifest/check_models/#manifest.check_models.CheckModelHasTags","title":"CheckModelHasTags","text":"

Models must have the specified tags.

Parameters:

Name Type Description Default model DbtBouncerModelBase

The DbtBouncerModelBase object to check.

required tags List[str]

List of tags to check for.

required

Other Parameters (passed via config file):

Name Type Description exclude Optional[str]

Regex pattern to match the model path. Model paths that match the pattern will not be checked.

include Optional[str]

Regex pattern to match the model path. Only model paths that match the pattern will be checked.

severity Optional[Literal['error', 'warn']]

Severity level of the check. Default: error.

Example(s):

manifest_checks:\n    - name: check_model_has_tags\n      tags:\n        - tag_1\n        - tag_2\n

Source code in src/dbt_bouncer/checks/manifest/check_models.py
class CheckModelHasTags(BaseCheck):\n    \"\"\"Models must have the specified tags.\n\n    Parameters:\n        model (DbtBouncerModelBase): The DbtBouncerModelBase object to check.\n        tags (List[str]): List of tags to check for.\n\n    Other Parameters:\n        exclude (Optional[str]): Regex pattern to match the model path. Model paths that match the pattern will not be checked.\n        include (Optional[str]): Regex pattern to match the model path. Only model paths that match the pattern will be checked.\n        severity (Optional[Literal[\"error\", \"warn\"]]): Severity level of the check. Default: `error`.\n\n    Example(s):\n        ```yaml\n        manifest_checks:\n            - name: check_model_has_tags\n              tags:\n                - tag_1\n                - tag_2\n        ```\n\n    \"\"\"\n\n    model: \"DbtBouncerModelBase\" = Field(default=None)\n    name: Literal[\"check_model_has_tags\"]\n    tags: List[str]\n\n    def execute(self) -> None:\n        \"\"\"Execute the check.\"\"\"\n        missing_tags = [tag for tag in self.tags if tag not in self.model.tags]\n        assert (\n            not missing_tags\n        ), f\"`{self.model.name}` is missing required tags: {missing_tags}.\"\n
"},{"location":"checks/manifest/check_models/#manifest.check_models.CheckModelHasUniqueTest","title":"CheckModelHasUniqueTest","text":"

Models must have a test for uniqueness of a column.

Parameters:

Name Type Description Default accepted_uniqueness_tests Optional[List[str]]

List of tests that are accepted as uniqueness tests.

required model DbtBouncerModelBase

The DbtBouncerModelBase object to check.

required tests List[DbtBouncerTestBase]

List of DbtBouncerTestBase objects parsed from manifest.json.

required

Other Parameters (passed via config file):

Name Type Description exclude Optional[str]

Regex pattern to match the model path. Model paths that match the pattern will not be checked.

include Optional[str]

Regex pattern to match the model path. Only model paths that match the pattern will be checked.

severity Optional[Literal['error', 'warn']]

Severity level of the check. Default: error.

Example(s):

manifest_checks:\n    - name: check_model_has_unique_test\n      include: ^models/marts\n
manifest_checks:\n# Example of allowing a custom uniqueness test\n    - name: check_model_has_unique_test\n      accepted_uniqueness_tests:\n        - expect_compound_columns_to_be_unique\n        - my_custom_uniqueness_test\n        - unique\n

Source code in src/dbt_bouncer/checks/manifest/check_models.py
class CheckModelHasUniqueTest(BaseCheck):\n    \"\"\"Models must have a test for uniqueness of a column.\n\n    Parameters:\n        accepted_uniqueness_tests (Optional[List[str]]): List of tests that are accepted as uniqueness tests.\n        model (DbtBouncerModelBase): The DbtBouncerModelBase object to check.\n        tests (List[DbtBouncerTestBase]): List of DbtBouncerTestBase objects parsed from `manifest.json`.\n\n    Other Parameters:\n        exclude (Optional[str]): Regex pattern to match the model path. Model paths that match the pattern will not be checked.\n        include (Optional[str]): Regex pattern to match the model path. Only model paths that match the pattern will be checked.\n        severity (Optional[Literal[\"error\", \"warn\"]]): Severity level of the check. Default: `error`.\n\n    Example(s):\n        ```yaml\n        manifest_checks:\n            - name: check_model_has_unique_test\n              include: ^models/marts\n        ```\n        ```yaml\n        manifest_checks:\n        # Example of allowing a custom uniqueness test\n            - name: check_model_has_unique_test\n              accepted_uniqueness_tests:\n                - expect_compound_columns_to_be_unique\n                - my_custom_uniqueness_test\n                - unique\n        ```\n\n    \"\"\"\n\n    accepted_uniqueness_tests: Optional[List[str]] = Field(\n        default=[\n            \"expect_compound_columns_to_be_unique\",\n            \"dbt_utils.unique_combination_of_columns\",\n            \"unique\",\n        ],\n    )\n    model: \"DbtBouncerModelBase\" = Field(default=None)\n    name: Literal[\"check_model_has_unique_test\"]\n    tests: List[\"DbtBouncerTestBase\"] = Field(default=[])\n\n    def execute(self) -> None:\n        \"\"\"Execute the check.\"\"\"\n        num_unique_tests = sum(\n            test.attached_node == self.model.unique_id\n            and test.test_metadata.name in self.accepted_uniqueness_tests  # type: ignore[operator]\n            for test in self.tests\n            if hasattr(test, \"test_metadata\")\n        )\n        assert (\n            num_unique_tests >= 1\n        ), f\"`{self.model.name}` does not have a test for uniqueness of a column.\"\n
"},{"location":"checks/manifest/check_models/#manifest.check_models.CheckModelHasUnitTests","title":"CheckModelHasUnitTests","text":"

Models must have more than the specified number of unit tests.

Parameters:

Name Type Description Default min_number_of_unit_tests Optional[int]

The minimum number of unit tests that a model must have.

required

Receives at execution time:

Name Type Description manifest_obj DbtBouncerManifest

The DbtBouncerManifest object parsed from manifest.json.

model DbtBouncerModelBase

The DbtBouncerModelBase object to check.

unit_tests List[UnitTests]

List of UnitTests objects parsed from manifest.json.

Other Parameters (passed via config file):

Name Type Description exclude Optional[str]

Regex pattern to match the model path. Model paths that match the pattern will not be checked.

include Optional[str]

Regex pattern to match the model path. Only model paths that match the pattern will be checked.

severity Optional[Literal['error', 'warn']]

Severity level of the check. Default: error.

Warning

This check is only supported for dbt 1.8.0 and above.

Example(s):

manifest_checks:\n    - name: check_model_has_unit_tests\n      include: ^models/marts\n
manifest_checks:\n    - name: check_model_has_unit_tests\n      min_number_of_unit_tests: 2\n

Source code in src/dbt_bouncer/checks/manifest/check_models.py
class CheckModelHasUnitTests(BaseCheck):\n    \"\"\"Models must have more than the specified number of unit tests.\n\n    Parameters:\n        min_number_of_unit_tests (Optional[int]): The minimum number of unit tests that a model must have.\n\n    Receives:\n        manifest_obj (DbtBouncerManifest): The DbtBouncerManifest object parsed from `manifest.json`.\n        model (DbtBouncerModelBase): The DbtBouncerModelBase object to check.\n        unit_tests (List[UnitTests]): List of UnitTests objects parsed from `manifest.json`.\n\n    Other Parameters:\n        exclude (Optional[str]): Regex pattern to match the model path. Model paths that match the pattern will not be checked.\n        include (Optional[str]): Regex pattern to match the model path. Only model paths that match the pattern will be checked.\n        severity (Optional[Literal[\"error\", \"warn\"]]): Severity level of the check. Default: `error`.\n\n    !!! warning\n\n        This check is only supported for dbt 1.8.0 and above.\n\n    Example(s):\n        ```yaml\n        manifest_checks:\n            - name: check_model_has_unit_tests\n              include: ^models/marts\n        ```\n        ```yaml\n        manifest_checks:\n            - name: check_model_has_unit_tests\n              min_number_of_unit_tests: 2\n        ```\n\n    \"\"\"\n\n    manifest_obj: \"DbtBouncerManifest\" = Field(default=None)\n    min_number_of_unit_tests: int = Field(default=1)\n    model: \"DbtBouncerModelBase\" = Field(default=None)\n    name: Literal[\"check_model_has_unit_tests\"]\n    unit_tests: List[\"UnitTests\"] = Field(default=[])\n\n    def execute(self) -> None:\n        \"\"\"Execute the check.\"\"\"\n        if (\n            semver.Version.parse(self.manifest_obj.manifest.metadata.dbt_version)\n            >= \"1.8.0\"\n        ):\n            num_unit_tests = len(\n                [\n                    t.unique_id\n                    for t in self.unit_tests\n                    if t.depends_on.nodes[0] == self.model.unique_id\n                ],\n            )\n            assert (\n                num_unit_tests >= self.min_number_of_unit_tests\n            ), f\"`{self.model.name}` has {num_unit_tests} unit tests, this is less than the minimum of {self.min_number_of_unit_tests}.\"\n        else:\n            logging.warning(\n                \"The `check_model_has_unit_tests` check is only supported for dbt 1.8.0 and above.\",\n            )\n
"},{"location":"checks/manifest/check_models/#manifest.check_models.CheckModelMaxChainedViews","title":"CheckModelMaxChainedViews","text":"

Models cannot have more than the specified number of upstream dependents that are not tables.

Parameters:

Name Type Description Default materializations_to_include Optional[List[str]]

List of materializations to include in the check.

required max_chained_views Optional[int]

The maximum number of upstream dependents that are not tables.

required

Receives at execution time:

Name Type Description model DbtBouncerModelBase

The DbtBouncerModelBase object to check.

models List[DbtBouncerModelBase]

List of DbtBouncerModelBase objects parsed from manifest.json.

Other Parameters (passed via config file):

Name Type Description exclude Optional[str]

Regex pattern to match the model path. Model paths that match the pattern will not be checked.

include Optional[str]

Regex pattern to match the model path. Only model paths that match the pattern will be checked.

severity Optional[Literal['error', 'warn']]

Severity level of the check. Default: error.

Example(s):

manifest_checks:\n    - name: check_model_max_chained_views\n
manifest_checks:\n    - name: check_model_max_chained_views\n      materializations_to_include:\n        - ephemeral\n        - my_custom_materialization\n        - view\n      max_chained_views: 5\n

Source code in src/dbt_bouncer/checks/manifest/check_models.py
class CheckModelMaxChainedViews(BaseCheck):\n    \"\"\"Models cannot have more than the specified number of upstream dependents that are not tables.\n\n    Parameters:\n        materializations_to_include (Optional[List[str]]): List of materializations to include in the check.\n        max_chained_views (Optional[int]): The maximum number of upstream dependents that are not tables.\n\n    Receives:\n        model (DbtBouncerModelBase): The DbtBouncerModelBase object to check.\n        models (List[DbtBouncerModelBase]): List of DbtBouncerModelBase objects parsed from `manifest.json`.\n\n    Other Parameters:\n        exclude (Optional[str]): Regex pattern to match the model path. Model paths that match the pattern will not be checked.\n        include (Optional[str]): Regex pattern to match the model path. Only model paths that match the pattern will be checked.\n        severity (Optional[Literal[\"error\", \"warn\"]]): Severity level of the check. Default: `error`.\n\n    Example(s):\n        ```yaml\n        manifest_checks:\n            - name: check_model_max_chained_views\n        ```\n        ```yaml\n        manifest_checks:\n            - name: check_model_max_chained_views\n              materializations_to_include:\n                - ephemeral\n                - my_custom_materialization\n                - view\n              max_chained_views: 5\n        ```\n\n    \"\"\"\n\n    manifest_obj: \"DbtBouncerManifest\" = Field(default=None)\n    materializations_to_include: List[str] = Field(\n        default=[\"ephemeral\", \"view\"],\n    )\n    max_chained_views: int = Field(\n        default=3,\n    )\n    model: \"DbtBouncerModelBase\" = Field(default=None)\n    models: List[\"DbtBouncerModelBase\"] = Field(default=[])\n    name: Literal[\"check_model_max_chained_views\"]\n\n    def execute(self) -> None:\n        \"\"\"Execute the check.\"\"\"\n\n        def return_upstream_view_models(\n            materializations,\n            max_chained_views,\n            models,\n            model_unique_ids_to_check,\n            package_name,\n            depth=0,\n        ):\n            \"\"\"Recursive function to return model unique_id's of upstream models that are views. Depth of recursion can be specified. If no models meet the criteria then an empty list is returned.\n\n            Returns\n            -\n                List[str]: List of model unique_id's of upstream models that are views.\n\n            \"\"\"\n            if depth == max_chained_views or model_unique_ids_to_check == []:\n                return model_unique_ids_to_check\n\n            relevant_upstream_models = []\n            for model in model_unique_ids_to_check:\n                upstream_nodes = list(\n                    next(m2 for m2 in models if m2.unique_id == model).depends_on.nodes,\n                )\n                if upstream_nodes != []:\n                    upstream_models = [\n                        m\n                        for m in upstream_nodes\n                        if m.split(\".\")[0] == \"model\"\n                        and m.split(\".\")[1] == package_name\n                    ]\n                    for i in upstream_models:\n                        if (\n                            next(\n                                m for m in models if m.unique_id == i\n                            ).config.materialized\n                            in materializations\n                        ):\n                            relevant_upstream_models.append(i)\n\n            depth += 1\n            return return_upstream_view_models(\n                materializations=materializations,\n                max_chained_views=max_chained_views,\n                models=models,\n                model_unique_ids_to_check=relevant_upstream_models,\n                package_name=package_name,\n                depth=depth,\n            )\n\n        assert (\n            len(\n                return_upstream_view_models(\n                    materializations=self.materializations_to_include,\n                    max_chained_views=self.max_chained_views,\n                    models=self.models,\n                    model_unique_ids_to_check=[self.model.unique_id],\n                    package_name=self.manifest_obj.manifest.metadata.project_name,\n                ),\n            )\n            == 0\n        ), f\"`{self.model.name}` has more than {self.max_chained_views} upstream dependents that are not tables.\"\n
"},{"location":"checks/manifest/check_models/#manifest.check_models.CheckModelMaxFanout","title":"CheckModelMaxFanout","text":"

Models cannot have more than the specified number of downstream models.

Parameters:

Name Type Description Default max_downstream_models Optional[int]

The maximum number of permitted downstream models.

required

Receives at execution time:

Name Type Description model DbtBouncerModelBase

The DbtBouncerModelBase object to check.

models List[DbtBouncerModelBase]

List of DbtBouncerModelBase objects parsed from manifest.json.

Other Parameters (passed via config file):

Name Type Description exclude Optional[str]

Regex pattern to match the model path. Model paths that match the pattern will not be checked.

include Optional[str]

Regex pattern to match the model path. Only model paths that match the pattern will be checked.

severity Optional[Literal['error', 'warn']]

Severity level of the check. Default: error.

Example(s):

manifest_checks:\n    - name: check_model_max_fanout\n      max_downstream_models: 2\n

Source code in src/dbt_bouncer/checks/manifest/check_models.py
class CheckModelMaxFanout(BaseCheck):\n    \"\"\"Models cannot have more than the specified number of downstream models.\n\n    Parameters:\n        max_downstream_models (Optional[int]): The maximum number of permitted downstream models.\n\n    Receives:\n        model (DbtBouncerModelBase): The DbtBouncerModelBase object to check.\n        models (List[DbtBouncerModelBase]): List of DbtBouncerModelBase objects parsed from `manifest.json`.\n\n    Other Parameters:\n        exclude (Optional[str]): Regex pattern to match the model path. Model paths that match the pattern will not be checked.\n        include (Optional[str]): Regex pattern to match the model path. Only model paths that match the pattern will be checked.\n        severity (Optional[Literal[\"error\", \"warn\"]]): Severity level of the check. Default: `error`.\n\n    Example(s):\n        ```yaml\n        manifest_checks:\n            - name: check_model_max_fanout\n              max_downstream_models: 2\n        ```\n\n    \"\"\"\n\n    max_downstream_models: int = Field(default=3)\n    model: \"DbtBouncerModelBase\" = Field(default=None)\n    models: List[\"DbtBouncerModelBase\"] = Field(default=[])\n    name: Literal[\"check_model_max_fanout\"]\n\n    def execute(self) -> None:\n        \"\"\"Execute the check.\"\"\"\n        num_downstream_models = sum(\n            self.model.unique_id in m.depends_on.nodes for m in self.models\n        )\n\n        assert (\n            num_downstream_models <= self.max_downstream_models\n        ), f\"`{self.model.name}` has {num_downstream_models} downstream models, which is more than the permitted maximum of {self.max_downstream_models}.\"\n
"},{"location":"checks/manifest/check_models/#manifest.check_models.CheckModelMaxNumberOfLines","title":"CheckModelMaxNumberOfLines","text":"

Models may not have more than the specified number of lines.

Parameters:

Name Type Description Default max_number_of_lines int

The maximum number of permitted lines.

required model DbtBouncerModelBase

The DbtBouncerModelBase object to check.

required

Other Parameters (passed via config file):

Name Type Description exclude Optional[str]

Regex pattern to match the model path. Model paths that match the pattern will not be checked.

include Optional[str]

Regex pattern to match the model path. Only model paths that match the pattern will be checked.

severity Optional[Literal['error', 'warn']]

Severity level of the check. Default: error.

Example(s):

manifest_checks:\n    - name: check_model_max_number_of_lines\n
manifest_checks:\n    - name: check_model_max_number_of_lines\n      max_number_of_lines: 150\n

Source code in src/dbt_bouncer/checks/manifest/check_models.py
class CheckModelMaxNumberOfLines(BaseCheck):\n    \"\"\"Models may not have more than the specified number of lines.\n\n    Parameters:\n        max_number_of_lines (int): The maximum number of permitted lines.\n\n        model (DbtBouncerModelBase): The DbtBouncerModelBase object to check.\n\n    Other Parameters:\n        exclude (Optional[str]): Regex pattern to match the model path. Model paths that match the pattern will not be checked.\n        include (Optional[str]): Regex pattern to match the model path. Only model paths that match the pattern will be checked.\n        severity (Optional[Literal[\"error\", \"warn\"]]): Severity level of the check. Default: `error`.\n\n    Example(s):\n        ```yaml\n        manifest_checks:\n            - name: check_model_max_number_of_lines\n        ```\n        ```yaml\n        manifest_checks:\n            - name: check_model_max_number_of_lines\n              max_number_of_lines: 150\n        ```\n\n    \"\"\"\n\n    model: \"DbtBouncerModelBase\" = Field(default=None)\n    name: Literal[\"check_model_max_number_of_lines\"]\n    max_number_of_lines: int = Field(default=100)\n\n    def execute(self) -> None:\n        \"\"\"Execute the check.\"\"\"\n        actual_number_of_lines = self.model.raw_code.count(\"\\n\") + 1\n\n        assert (\n            actual_number_of_lines <= self.max_number_of_lines\n        ), f\"`{self.model.name}` has {actual_number_of_lines} lines, this is more than the maximum permitted number of lines ({self.max_number_of_lines}).\"\n
"},{"location":"checks/manifest/check_models/#manifest.check_models.CheckModelMaxUpstreamDependencies","title":"CheckModelMaxUpstreamDependencies","text":"

Limit the number of upstream dependencies a model has.

Parameters:

Name Type Description Default max_upstream_macros Optional[int]

The maximum number of permitted upstream macros.

required max_upstream_models Optional[int]

The maximum number of permitted upstream models.

required max_upstream_sources Optional[int]

The maximum number of permitted upstream sources.

required

Receives at execution time:

Name Type Description model DbtBouncerModelBase

The DbtBouncerModelBase object to check.

Other Parameters (passed via config file):

Name Type Description exclude Optional[str]

Regex pattern to match the model path. Model paths that match the pattern will not be checked.

include Optional[str]

Regex pattern to match the model path. Only model paths that match the pattern will be checked.

severity Optional[Literal['error', 'warn']]

Severity level of the check. Default: error.

Example(s):

manifest_checks:\n    - name: check_model_max_upstream_dependencies\n      max_upstream_models: 3\n

Source code in src/dbt_bouncer/checks/manifest/check_models.py
class CheckModelMaxUpstreamDependencies(BaseCheck):\n    \"\"\"Limit the number of upstream dependencies a model has.\n\n    Parameters:\n        max_upstream_macros (Optional[int]): The maximum number of permitted upstream macros.\n        max_upstream_models (Optional[int]): The maximum number of permitted upstream models.\n        max_upstream_sources (Optional[int]): The maximum number of permitted upstream sources.\n\n    Receives:\n        model (DbtBouncerModelBase): The DbtBouncerModelBase object to check.\n\n    Other Parameters:\n        exclude (Optional[str]): Regex pattern to match the model path. Model paths that match the pattern will not be checked.\n        include (Optional[str]): Regex pattern to match the model path. Only model paths that match the pattern will be checked.\n        severity (Optional[Literal[\"error\", \"warn\"]]): Severity level of the check. Default: `error`.\n\n    Example(s):\n        ```yaml\n        manifest_checks:\n            - name: check_model_max_upstream_dependencies\n              max_upstream_models: 3\n        ```\n\n    \"\"\"\n\n    max_upstream_macros: int = Field(\n        default=5,\n    )\n    max_upstream_models: int = Field(\n        default=5,\n    )\n    max_upstream_sources: int = Field(\n        default=1,\n    )\n    model: \"DbtBouncerModelBase\" = Field(default=None)\n    name: Literal[\"check_model_max_upstream_dependencies\"]\n\n    def execute(self) -> None:\n        \"\"\"Execute the check.\"\"\"\n        num_upstream_macros = len(list(self.model.depends_on.macros))\n        num_upstream_models = len(\n            [m for m in self.model.depends_on.nodes if m.split(\".\")[0] == \"model\"],\n        )\n        num_upstream_sources = len(\n            [m for m in self.model.depends_on.nodes if m.split(\".\")[0] == \"source\"],\n        )\n\n        assert (\n            num_upstream_macros <= self.max_upstream_macros\n        ), f\"`{self.model.name}` has {num_upstream_macros} upstream macros, which is more than the permitted maximum of {self.max_upstream_macros}.\"\n        assert (\n            num_upstream_models <= self.max_upstream_models\n        ), f\"`{self.model.name}` has {num_upstream_models} upstream models, which is more than the permitted maximum of {self.max_upstream_models}.\"\n        assert (\n            num_upstream_sources <= self.max_upstream_sources\n        ), f\"`{self.model.name}` has {num_upstream_sources} upstream sources, which is more than the permitted maximum of {self.max_upstream_sources}.\"\n
"},{"location":"checks/manifest/check_models/#manifest.check_models.CheckModelNames","title":"CheckModelNames","text":"

Models must have a name that matches the supplied regex.

Parameters:

Name Type Description Default model_name_pattern str

Regexp the model name must match.

required

Receives at execution time:

Name Type Description model DbtBouncerModelBase

The DbtBouncerModelBase object to check.

Other Parameters (passed via config file):

Name Type Description exclude Optional[str]

Regex pattern to match the model path. Model paths that match the pattern will not be checked.

include Optional[str]

Regex pattern to match the model path. Only model paths that match the pattern will be checked.

severity Optional[Literal['error', 'warn']]

Severity level of the check. Default: error.

Example(s):

manifest_checks:\n    - name: check_model_names\n      include: ^models/intermediate\n      model_name_pattern: ^int_\n    - name: check_model_names\n      include: ^models/staging\n      model_name_pattern: ^stg_\n

Source code in src/dbt_bouncer/checks/manifest/check_models.py
class CheckModelNames(BaseCheck):\n    \"\"\"Models must have a name that matches the supplied regex.\n\n    Parameters:\n        model_name_pattern (str): Regexp the model name must match.\n\n    Receives:\n        model (DbtBouncerModelBase): The DbtBouncerModelBase object to check.\n\n    Other Parameters:\n        exclude (Optional[str]): Regex pattern to match the model path. Model paths that match the pattern will not be checked.\n        include (Optional[str]): Regex pattern to match the model path. Only model paths that match the pattern will be checked.\n        severity (Optional[Literal[\"error\", \"warn\"]]): Severity level of the check. Default: `error`.\n\n    Example(s):\n        ```yaml\n        manifest_checks:\n            - name: check_model_names\n              include: ^models/intermediate\n              model_name_pattern: ^int_\n            - name: check_model_names\n              include: ^models/staging\n              model_name_pattern: ^stg_\n        ```\n\n    \"\"\"\n\n    model_config = ConfigDict(extra=\"forbid\", protected_namespaces=())\n\n    model: \"DbtBouncerModelBase\" = Field(default=None)\n    name: Literal[\"check_model_names\"]\n    model_name_pattern: str\n\n    def execute(self) -> None:\n        \"\"\"Execute the check.\"\"\"\n        assert (\n            re.compile(self.model_name_pattern.strip()).match(self.model.name)\n            is not None\n        ), f\"`{self.model.name}` does not match the supplied regex `{self.model_name_pattern.strip()})`.\"\n
"},{"location":"checks/manifest/check_models/#manifest.check_models.CheckModelPropertyFileLocation","title":"CheckModelPropertyFileLocation","text":"

Model properties files must follow the guidance provided by dbt here.

Parameters:

Name Type Description Default model DbtBouncerModelBase

The DbtBouncerModelBase object to check.

required

Other Parameters (passed via config file):

Name Type Description exclude Optional[str]

Regex pattern to match the model path. Model paths that match the pattern will not be checked.

include Optional[str]

Regex pattern to match the model path. Only model paths that match the pattern will be checked.

severity Optional[Literal['error', 'warn']]

Severity level of the check. Default: error.

Example(s):

manifest_checks:\n    - name: check_model_property_file_location\n

Source code in src/dbt_bouncer/checks/manifest/check_models.py
class CheckModelPropertyFileLocation(BaseCheck):\n    \"\"\"Model properties files must follow the guidance provided by dbt [here](https://docs.getdbt.com/best-practices/how-we-structure/1-guide-overview).\n\n    Parameters:\n        model (DbtBouncerModelBase): The DbtBouncerModelBase object to check.\n\n    Other Parameters:\n        exclude (Optional[str]): Regex pattern to match the model path. Model paths that match the pattern will not be checked.\n        include (Optional[str]): Regex pattern to match the model path. Only model paths that match the pattern will be checked.\n        severity (Optional[Literal[\"error\", \"warn\"]]): Severity level of the check. Default: `error`.\n\n    Example(s):\n        ```yaml\n        manifest_checks:\n            - name: check_model_property_file_location\n        ```\n\n    \"\"\"\n\n    model: \"DbtBouncerModelBase\" = Field(default=None)\n    name: Literal[\"check_model_property_file_location\"]\n\n    def execute(self) -> None:\n        \"\"\"Execute the check.\"\"\"\n        assert (  # noqa: PT018\n            hasattr(self.model, \"patch_path\")\n            and clean_path_str(self.model.patch_path) is not None\n        ), f\"`{self.model.name}` is not documented.\"\n\n        expected_substr = (\n            \"_\".join(clean_path_str(self.model.original_file_path).split(\"/\")[1:-1])\n            .replace(\"staging\", \"stg\")\n            .replace(\"intermediate\", \"int\")\n            .replace(\"marts\", \"\")\n        )\n        properties_yml_name = clean_path_str(self.model.patch_path).split(\"/\")[-1]\n\n        assert properties_yml_name.startswith(\n            \"_\",\n        ), f\"The properties file for `{self.model.name}` (`{properties_yml_name}`) does not start with an underscore.\"\n        assert (\n            expected_substr in properties_yml_name\n        ), f\"The properties file for `{self.model.name}` (`{properties_yml_name}`) does not contain the expected substring (`{expected_substr}`).\"\n        assert properties_yml_name.endswith(\n            \"__models.yml\",\n        ), f\"The properties file for `{self.model.name}` (`{properties_yml_name}`) does not end with `__models.yml`.\"\n
"},{"location":"checks/manifest/check_models/#manifest.check_models.CheckModelsTestCoverage","title":"CheckModelsTestCoverage","text":"

Set the minimum percentage of models that have at least one test.

Parameters:

Name Type Description Default min_model_test_coverage_pct float

The minimum percentage of models that must have at least one test.

required models List[DbtBouncerModelBase]

List of DbtBouncerModelBase objects parsed from manifest.json.

required tests List[DbtBouncerTestBase]

List of DbtBouncerTestBase objects parsed from manifest.json.

required

Other Parameters (passed via config file):

Name Type Description severity Optional[Literal['error', 'warn']]

Severity level of the check. Default: error.

Example(s):

manifest_checks:\n    - name: check_model_test_coverage\n      min_model_test_coverage_pct: 90\n

Source code in src/dbt_bouncer/checks/manifest/check_models.py
class CheckModelsTestCoverage(BaseModel):\n    \"\"\"Set the minimum percentage of models that have at least one test.\n\n    Parameters:\n        min_model_test_coverage_pct (float): The minimum percentage of models that must have at least one test.\n        models (List[DbtBouncerModelBase]): List of DbtBouncerModelBase objects parsed from `manifest.json`.\n        tests (List[DbtBouncerTestBase]): List of DbtBouncerTestBase objects parsed from `manifest.json`.\n\n    Other Parameters:\n        severity (Optional[Literal[\"error\", \"warn\"]]): Severity level of the check. Default: `error`.\n\n\n    Example(s):\n        ```yaml\n        manifest_checks:\n            - name: check_model_test_coverage\n              min_model_test_coverage_pct: 90\n        ```\n\n    \"\"\"\n\n    model_config = ConfigDict(extra=\"forbid\")\n\n    index: Optional[int] = Field(\n        default=None,\n        description=\"Index to uniquely identify the check, calculated at runtime.\",\n    )\n    name: Literal[\"check_model_test_coverage\"]\n    min_model_test_coverage_pct: float = Field(\n        default=100,\n        ge=0,\n        le=100,\n    )\n    models: List[\"DbtBouncerModelBase\"] = Field(default=[])\n    severity: Optional[Literal[\"error\", \"warn\"]] = Field(\n        default=\"error\",\n        description=\"Severity of the check, one of 'error' or 'warn'.\",\n    )\n    tests: List[\"DbtBouncerTestBase\"] = Field(default=[])\n\n    def execute(self) -> None:\n        \"\"\"Execute the check.\"\"\"\n        num_models = len(self.models)\n        models_with_tests = []\n        for model in self.models:\n            for test in self.tests:\n                if model.unique_id in test.depends_on.nodes:\n                    models_with_tests.append(model.unique_id)\n        num_models_with_tests = len(set(models_with_tests))\n        model_test_coverage_pct = (num_models_with_tests / num_models) * 100\n\n        assert (\n            model_test_coverage_pct >= self.min_model_test_coverage_pct\n        ), f\"Only {model_test_coverage_pct}% of models have at least one test, this is less than the permitted minimum of {self.min_model_test_coverage_pct}%.\"\n
"},{"location":"checks/manifest/check_semantic_models/","title":"Manifest Checks: Semantic Models","text":"

Note

The below checks require manifest.json to be present.

"},{"location":"checks/manifest/check_semantic_models/#manifest.check_semantic_models.CheckSemanticModelOnNonPublicModels","title":"CheckSemanticModelOnNonPublicModels","text":"

Semantic models should be based on public models only.

Receives at execution time:

Name Type Description models List[DbtBouncerModelBase]

List of DbtBouncerModelBase objects parsed from manifest.json.

semantic_model DbtBouncerSemanticModelBase

The DbtBouncerSemanticModelBase object to check.

Other Parameters (passed via config file):

Name Type Description exclude Optional[str]

Regex pattern to match the semantic model path (i.e the .yml file where the semantic model is configured). Semantic model paths that match the pattern will not be checked.

include Optional[str]

Regex pattern to match the semantic model path (i.e the .yml file where the semantic model is configured). Only semantic model paths that match the pattern will be checked.

severity Optional[Literal['error', 'warn']]

Severity level of the check. Default: error.

Example(s):

manifest_checks:\n    - name: check_semantic_model_based_on_non_public_models\n

Source code in src/dbt_bouncer/checks/manifest/check_semantic_models.py
class CheckSemanticModelOnNonPublicModels(BaseCheck):\n    \"\"\"Semantic models should be based on public models only.\n\n    Receives:\n        models (List[DbtBouncerModelBase]): List of DbtBouncerModelBase objects parsed from `manifest.json`.\n        semantic_model (DbtBouncerSemanticModelBase): The DbtBouncerSemanticModelBase object to check.\n\n    Other Parameters:\n        exclude (Optional[str]): Regex pattern to match the semantic model path (i.e the .yml file where the semantic model is configured). Semantic model paths that match the pattern will not be checked.\n        include (Optional[str]): Regex pattern to match the semantic model path (i.e the .yml file where the semantic model is configured). Only semantic model paths that match the pattern will be checked.\n        severity (Optional[Literal[\"error\", \"warn\"]]): Severity level of the check. Default: `error`.\n\n    Example(s):\n        ```yaml\n        manifest_checks:\n            - name: check_semantic_model_based_on_non_public_models\n        ```\n\n    \"\"\"\n\n    models: List[\"DbtBouncerModelBase\"] = Field(default=[])\n    name: Literal[\"check_semantic_model_based_on_non_public_models\"]\n    semantic_model: \"DbtBouncerSemanticModelBase\" = Field(default=None)\n\n    def execute(self) -> None:\n        \"\"\"Execute the check.\"\"\"\n        non_public_upstream_dependencies = []\n        for model in self.semantic_model.depends_on.nodes:\n            if (\n                model.split(\".\")[0] == \"model\"\n                and model.split(\".\")[1] == self.semantic_model.package_name\n            ):\n                model = next(m for m in self.models if m.unique_id == model)\n                if model.access.value != \"public\":\n                    non_public_upstream_dependencies.append(model.name)\n\n        assert not non_public_upstream_dependencies, f\"Semantic model `{self.semantic_model.name}` is based on a model(s) that is not public: {non_public_upstream_dependencies}.\"\n
"},{"location":"checks/manifest/check_sources/","title":"Manifest Checks: Sources","text":"

Note

The below checks require manifest.json to be present.

"},{"location":"checks/manifest/check_sources/#manifest.check_sources.CheckSourceDescriptionPopulated","title":"CheckSourceDescriptionPopulated","text":"

Sources must have a populated description.

Receives at execution time:

Name Type Description source DbtBouncerSourceBase

The DbtBouncerSourceBase object to check.

Other Parameters (passed via config file):

Name Type Description exclude Optional[str]

Regex pattern to match the source path (i.e the .yml file where the source is configured). Source paths that match the pattern will not be checked.

include Optional[str]

Regex pattern to match the source path (i.e the .yml file where the source is configured). Only source paths that match the pattern will be checked.

severity Optional[Literal['error', 'warn']]

Severity level of the check. Default: error.

Example(s):

manifest_checks:\n    - name: check_source_description_populated\n

Source code in src/dbt_bouncer/checks/manifest/check_sources.py
class CheckSourceDescriptionPopulated(BaseCheck):\n    \"\"\"Sources must have a populated description.\n\n    Receives:\n        source (DbtBouncerSourceBase): The DbtBouncerSourceBase object to check.\n\n    Other Parameters:\n        exclude (Optional[str]): Regex pattern to match the source path (i.e the .yml file where the source is configured). Source paths that match the pattern will not be checked.\n        include (Optional[str]): Regex pattern to match the source path (i.e the .yml file where the source is configured). Only source paths that match the pattern will be checked.\n        severity (Optional[Literal[\"error\", \"warn\"]]): Severity level of the check. Default: `error`.\n\n    Example(s):\n        ```yaml\n        manifest_checks:\n            - name: check_source_description_populated\n        ```\n\n    \"\"\"\n\n    name: Literal[\"check_source_description_populated\"]\n    source: \"DbtBouncerSourceBase\" = Field(default=None)\n\n    def execute(self) -> None:\n        \"\"\"Execute the check.\"\"\"\n        assert (\n            len(self.source.description.strip()) > 4\n        ), f\"`{self.source.source_name}.{self.source.name}` does not have a populated description.\"\n
"},{"location":"checks/manifest/check_sources/#manifest.check_sources.CheckSourceFreshnessPopulated","title":"CheckSourceFreshnessPopulated","text":"

Sources must have a populated freshness.

Receives at execution time:

Name Type Description source DbtBouncerSource

The DbtBouncerSourceBase object to check.

Other Parameters (passed via config file):

Name Type Description exclude Optional[str]

Regex pattern to match the source path (i.e the .yml file where the source is configured). Source paths that match the pattern will not be checked.

include Optional[str]

Regex pattern to match the source path (i.e the .yml file where the source is configured). Only source paths that match the pattern will be checked.

severity Optional[Literal['error', 'warn']]

Severity level of the check. Default: error.

Example(s):

manifest_checks:\n    - name: check_source_freshness_populated\n

Source code in src/dbt_bouncer/checks/manifest/check_sources.py
class CheckSourceFreshnessPopulated(BaseCheck):\n    \"\"\"Sources must have a populated freshness.\n\n    Receives:\n        source (DbtBouncerSource): The DbtBouncerSourceBase object to check.\n\n    Other Parameters:\n        exclude (Optional[str]): Regex pattern to match the source path (i.e the .yml file where the source is configured). Source paths that match the pattern will not be checked.\n        include (Optional[str]): Regex pattern to match the source path (i.e the .yml file where the source is configured). Only source paths that match the pattern will be checked.\n        severity (Optional[Literal[\"error\", \"warn\"]]): Severity level of the check. Default: `error`.\n\n    Example(s):\n        ```yaml\n        manifest_checks:\n            - name: check_source_freshness_populated\n        ```\n\n    \"\"\"\n\n    name: Literal[\"check_source_freshness_populated\"]\n    source: \"DbtBouncerSourceBase\" = Field(default=None)\n\n    def execute(self) -> None:\n        \"\"\"Execute the check.\"\"\"\n        error_msg = f\"`{self.source.source_name}.{self.source.name}` does not have a populated freshness.\"\n        assert self.source.freshness is not None, error_msg\n        assert (\n            self.source.freshness.error_after.count is not None\n            and self.source.freshness.error_after.period is not None\n        ) or (\n            self.source.freshness.warn_after.count is not None\n            and self.source.freshness.warn_after.period is not None\n        ), error_msg\n
"},{"location":"checks/manifest/check_sources/#manifest.check_sources.CheckSourceHasMetaKeys","title":"CheckSourceHasMetaKeys","text":"

The meta config for sources must have the specified keys.

Parameters:

Name Type Description Default keys NestedDict

A list (that may contain sub-lists) of required keys.

required

Receives at execution time:

Name Type Description source DbtBouncerSource

The DbtBouncerSourceBase object to check.

Other Parameters (passed via config file):

Name Type Description exclude Optional[str]

Regex pattern to match the source path (i.e the .yml file where the source is configured). Source paths that match the pattern will not be checked.

include Optional[str]

Regex pattern to match the source path (i.e the .yml file where the source is configured). Only source paths that match the pattern will be checked.

severity Optional[Literal['error', 'warn']]

Severity level of the check. Default: error.

Example(s):

manifest_checks:\n    - name: check_source_has_meta_keys\n      keys:\n        - contact:\n            - email\n            - slack\n        - owner\n

Source code in src/dbt_bouncer/checks/manifest/check_sources.py
class CheckSourceHasMetaKeys(BaseCheck):\n    \"\"\"The `meta` config for sources must have the specified keys.\n\n    Parameters:\n        keys (NestedDict): A list (that may contain sub-lists) of required keys.\n\n    Receives:\n        source (DbtBouncerSource): The DbtBouncerSourceBase object to check.\n\n    Other Parameters:\n        exclude (Optional[str]): Regex pattern to match the source path (i.e the .yml file where the source is configured). Source paths that match the pattern will not be checked.\n        include (Optional[str]): Regex pattern to match the source path (i.e the .yml file where the source is configured). Only source paths that match the pattern will be checked.\n        severity (Optional[Literal[\"error\", \"warn\"]]): Severity level of the check. Default: `error`.\n\n    Example(s):\n        ```yaml\n        manifest_checks:\n            - name: check_source_has_meta_keys\n              keys:\n                - contact:\n                    - email\n                    - slack\n                - owner\n        ```\n\n    \"\"\"\n\n    keys: \"NestedDict\"\n    name: Literal[\"check_source_has_meta_keys\"]\n    source: \"DbtBouncerSourceBase\" = Field(default=None)\n\n    def execute(self) -> None:\n        \"\"\"Execute the check.\"\"\"\n        missing_keys = find_missing_meta_keys(\n            meta_config=self.source.meta,\n            required_keys=self.keys.model_dump(),\n        )\n\n        assert (\n            missing_keys == []\n        ), f\"`{self.source.source_name}.{self.source.name}` is missing the following keys from the `meta` config: {[x.replace('>>', '') for x in missing_keys]}\"\n
"},{"location":"checks/manifest/check_sources/#manifest.check_sources.CheckSourceHasTags","title":"CheckSourceHasTags","text":"

Sources must have the specified tags.

Parameters:

Name Type Description Default source DbtBouncerSource

The DbtBouncerSourceBase object to check.

required tags List[str]

List of tags to check for.

required

Other Parameters (passed via config file):

Name Type Description exclude Optional[str]

Regex pattern to match the source path (i.e the .yml file where the source is configured). Source paths that match the pattern will not be checked.

include Optional[str]

Regex pattern to match the source path (i.e the .yml file where the source is configured). Only source paths that match the pattern will be checked.

severity Optional[Literal['error', 'warn']]

Severity level of the check. Default: error.

Example(s):

manifest_checks:\n    - name: check_source_has_tags\n      tags:\n        - tag_1\n        - tag_2\n

Source code in src/dbt_bouncer/checks/manifest/check_sources.py
class CheckSourceHasTags(BaseCheck):\n    \"\"\"Sources must have the specified tags.\n\n    Parameters:\n        source (DbtBouncerSource): The DbtBouncerSourceBase object to check.\n        tags (List[str]): List of tags to check for.\n\n    Other Parameters:\n        exclude (Optional[str]): Regex pattern to match the source path (i.e the .yml file where the source is configured). Source paths that match the pattern will not be checked.\n        include (Optional[str]): Regex pattern to match the source path (i.e the .yml file where the source is configured). Only source paths that match the pattern will be checked.\n        severity (Optional[Literal[\"error\", \"warn\"]]): Severity level of the check. Default: `error`.\n\n    Example(s):\n        ```yaml\n        manifest_checks:\n            - name: check_source_has_tags\n              tags:\n                - tag_1\n                - tag_2\n        ```\n\n    \"\"\"\n\n    name: Literal[\"check_source_has_tags\"]\n    source: \"DbtBouncerSourceBase\" = Field(default=None)\n    tags: List[str]\n\n    def execute(self) -> None:\n        \"\"\"Execute the check.\"\"\"\n        missing_tags = [tag for tag in self.tags if tag not in self.source.tags]\n        assert not missing_tags, f\"`{self.source.source_name}.{self.source.name}` is missing required tags: {missing_tags}.\"\n
"},{"location":"checks/manifest/check_sources/#manifest.check_sources.CheckSourceLoaderPopulated","title":"CheckSourceLoaderPopulated","text":"

Sources must have a populated loader.

Parameters:

Name Type Description Default source DbtBouncerSource

The DbtBouncerSourceBase object to check.

required

Other Parameters (passed via config file):

Name Type Description exclude Optional[str]

Regex pattern to match the source path (i.e the .yml file where the source is configured). Source paths that match the pattern will not be checked.

include Optional[str]

Regex pattern to match the source path (i.e the .yml file where the source is configured). Only source paths that match the pattern will be checked.

severity Optional[Literal['error', 'warn']]

Severity level of the check. Default: error.

Example(s):

manifest_checks:\n    - name: check_source_loader_populated\n

Source code in src/dbt_bouncer/checks/manifest/check_sources.py
class CheckSourceLoaderPopulated(BaseCheck):\n    \"\"\"Sources must have a populated loader.\n\n    Parameters:\n        source (DbtBouncerSource): The DbtBouncerSourceBase object to check.\n\n    Other Parameters:\n        exclude (Optional[str]): Regex pattern to match the source path (i.e the .yml file where the source is configured). Source paths that match the pattern will not be checked.\n        include (Optional[str]): Regex pattern to match the source path (i.e the .yml file where the source is configured). Only source paths that match the pattern will be checked.\n        severity (Optional[Literal[\"error\", \"warn\"]]): Severity level of the check. Default: `error`.\n\n    Example(s):\n        ```yaml\n        manifest_checks:\n            - name: check_source_loader_populated\n        ```\n\n    \"\"\"\n\n    name: Literal[\"check_source_loader_populated\"]\n    source: \"DbtBouncerSourceBase\" = Field(default=None)\n\n    def execute(self) -> None:\n        \"\"\"Execute the check.\"\"\"\n        assert (\n            self.source.loader != \"\"\n        ), f\"`{self.source.source_name}.{self.source.name}` does not have a populated loader.\"\n
"},{"location":"checks/manifest/check_sources/#manifest.check_sources.CheckSourceNames","title":"CheckSourceNames","text":"

Sources must have a name that matches the supplied regex.

Parameters:

Name Type Description Default source_name_pattern str

Regexp the source name must match.

required

Receives at execution time:

Name Type Description source DbtBouncerSource

The DbtBouncerSourceBase object to check.

Other Parameters (passed via config file):

Name Type Description exclude Optional[str]

Regex pattern to match the source path (i.e the .yml file where the source is configured). Source paths that match the pattern will not be checked.

include Optional[str]

Regex pattern to match the source path (i.e the .yml file where the source is configured). Only source paths that match the pattern will be checked.

severity Optional[Literal['error', 'warn']]

Severity level of the check. Default: error.

Example(s):

manifest_checks:\n    - name: check_source_names\n      source_name_pattern: >\n        ^[a-z0-9_]*$\n

Source code in src/dbt_bouncer/checks/manifest/check_sources.py
class CheckSourceNames(BaseCheck):\n    \"\"\"Sources must have a name that matches the supplied regex.\n\n    Parameters:\n        source_name_pattern (str): Regexp the source name must match.\n\n    Receives:\n        source (DbtBouncerSource): The DbtBouncerSourceBase object to check.\n\n    Other Parameters:\n        exclude (Optional[str]): Regex pattern to match the source path (i.e the .yml file where the source is configured). Source paths that match the pattern will not be checked.\n        include (Optional[str]): Regex pattern to match the source path (i.e the .yml file where the source is configured). Only source paths that match the pattern will be checked.\n        severity (Optional[Literal[\"error\", \"warn\"]]): Severity level of the check. Default: `error`.\n\n    Example(s):\n        ```yaml\n        manifest_checks:\n            - name: check_source_names\n              source_name_pattern: >\n                ^[a-z0-9_]*$\n        ```\n\n    \"\"\"\n\n    name: Literal[\"check_source_names\"]\n    source_name_pattern: str\n    source: \"DbtBouncerSourceBase\" = Field(default=None)\n\n    def execute(self) -> None:\n        \"\"\"Execute the check.\"\"\"\n        assert (\n            re.compile(self.source_name_pattern.strip()).match(self.source.name)\n            is not None\n        ), f\"`{self.source.source_name}.{self.source.name}` does not match the supplied regex `({self.source_name_pattern.strip()})`.\"\n
"},{"location":"checks/manifest/check_sources/#manifest.check_sources.CheckSourceNotOrphaned","title":"CheckSourceNotOrphaned","text":"

Sources must be referenced in at least one model.

Receives at execution time:

Name Type Description models List[DbtBouncerModelBase]

List of DbtBouncerModelBase objects parsed from manifest.json.

source DbtBouncerSource

The DbtBouncerSourceBase object to check.

Other Parameters (passed via config file):

Name Type Description exclude Optional[str]

Regex pattern to match the source path (i.e the .yml file where the source is configured). Source paths that match the pattern will not be checked.

include Optional[str]

Regex pattern to match the source path (i.e the .yml file where the source is configured). Only source paths that match the pattern will be checked.

severity Optional[Literal['error', 'warn']]

Severity level of the check. Default: error.

Example(s):

manifest_checks:\n    - name: check_source_not_orphaned\n

Source code in src/dbt_bouncer/checks/manifest/check_sources.py
class CheckSourceNotOrphaned(BaseCheck):\n    \"\"\"Sources must be referenced in at least one model.\n\n    Receives:\n        models (List[DbtBouncerModelBase]): List of DbtBouncerModelBase objects parsed from `manifest.json`.\n        source (DbtBouncerSource): The DbtBouncerSourceBase object to check.\n\n    Other Parameters:\n        exclude (Optional[str]): Regex pattern to match the source path (i.e the .yml file where the source is configured). Source paths that match the pattern will not be checked.\n        include (Optional[str]): Regex pattern to match the source path (i.e the .yml file where the source is configured). Only source paths that match the pattern will be checked.\n        severity (Optional[Literal[\"error\", \"warn\"]]): Severity level of the check. Default: `error`.\n\n    Example(s):\n        ```yaml\n        manifest_checks:\n            - name: check_source_not_orphaned\n        ```\n\n    \"\"\"\n\n    models: List[\"DbtBouncerModelBase\"] = Field(default=[])\n    name: Literal[\"check_source_not_orphaned\"]\n    source: \"DbtBouncerSourceBase\" = Field(default=None)\n\n    def execute(self) -> None:\n        \"\"\"Execute the check.\"\"\"\n        num_refs = sum(\n            self.source.unique_id in model.depends_on.nodes for model in self.models\n        )\n        assert (\n            num_refs >= 1\n        ), f\"Source `{self.source.source_name}.{self.source.name}` is orphaned, i.e. not referenced by any model.\"\n
"},{"location":"checks/manifest/check_sources/#manifest.check_sources.CheckSourcePropertyFileLocation","title":"CheckSourcePropertyFileLocation","text":"

Source properties files must follow the guidance provided by dbt here.

Receives at execution time:

Name Type Description source DbtBouncerSource

The DbtBouncerSourceBase object to check.

Other Parameters (passed via config file):

Name Type Description exclude Optional[str]

Regex pattern to match the source path (i.e the .yml file where the source is configured). Source paths that match the pattern will not be checked.

include Optional[str]

Regex pattern to match the source path (i.e the .yml file where the source is configured). Only source paths that match the pattern will be checked.

severity Optional[Literal['error', 'warn']]

Severity level of the check. Default: error.

Example(s):

manifest_checks:\n    - name: check_source_property_file_location\n

Source code in src/dbt_bouncer/checks/manifest/check_sources.py
class CheckSourcePropertyFileLocation(BaseCheck):\n    \"\"\"Source properties files must follow the guidance provided by dbt [here](https://docs.getdbt.com/best-practices/how-we-structure/1-guide-overview).\n\n    Receives:\n        source (DbtBouncerSource): The DbtBouncerSourceBase object to check.\n\n    Other Parameters:\n        exclude (Optional[str]): Regex pattern to match the source path (i.e the .yml file where the source is configured). Source paths that match the pattern will not be checked.\n        include (Optional[str]): Regex pattern to match the source path (i.e the .yml file where the source is configured). Only source paths that match the pattern will be checked.\n        severity (Optional[Literal[\"error\", \"warn\"]]): Severity level of the check. Default: `error`.\n\n    Example(s):\n        ```yaml\n        manifest_checks:\n            - name: check_source_property_file_location\n        ```\n\n    \"\"\"\n\n    name: Literal[\"check_source_property_file_location\"]\n    source: \"DbtBouncerSourceBase\" = Field(default=None)\n\n    def execute(self) -> None:\n        \"\"\"Execute the check.\"\"\"\n        path_cleaned = clean_path_str(self.source.original_file_path).replace(\n            \"models/staging\", \"\"\n        )\n        expected_substring = \"_\".join(path_cleaned.split(\"/\")[:-1])\n\n        assert path_cleaned.split(\n            \"/\",\n        )[\n            -1\n        ].startswith(\n            \"_\",\n        ), f\"The properties file for `{self.source.source_name}.{self.source.name}` (`{path_cleaned}`) does not start with an underscore.\"\n        assert (\n            expected_substring in path_cleaned\n        ), f\"The properties file for `{self.source.source_name}.{self.source.name}` (`{path_cleaned}`) does not contain the expected substring (`{expected_substring}`).\"\n        assert path_cleaned.split(\n            \"/\",\n        )[\n            -1\n        ].endswith(\n            \"__sources.yml\",\n        ), f\"The properties file for `{self.source.source_name}.{self.source.name}` (`{path_cleaned}`) does not end with `__sources.yml`.\"\n
"},{"location":"checks/manifest/check_sources/#manifest.check_sources.CheckSourceUsedByModelsInSameDirectory","title":"CheckSourceUsedByModelsInSameDirectory","text":"

Sources can only be referenced by models that are located in the same directory where the source is defined.

Parameters:

Name Type Description Default models List[DbtBouncerModelBase]

List of DbtBouncerModelBase objects parsed from manifest.json.

required source DbtBouncerSource

The DbtBouncerSourceBase object to check.

required

Other Parameters (passed via config file):

Name Type Description exclude Optional[str]

Regex pattern to match the source path (i.e the .yml file where the source is configured). Source paths that match the pattern will not be checked.

include Optional[str]

Regex pattern to match the source path (i.e the .yml file where the source is configured). Only source paths that match the pattern will be checked.

severity Optional[Literal['error', 'warn']]

Severity level of the check. Default: error.

Example(s):

manifest_checks:\n    - name: check_source_used_by_models_in_same_directory\n

Source code in src/dbt_bouncer/checks/manifest/check_sources.py
class CheckSourceUsedByModelsInSameDirectory(BaseCheck):\n    \"\"\"Sources can only be referenced by models that are located in the same directory where the source is defined.\n\n    Parameters:\n        models (List[DbtBouncerModelBase]): List of DbtBouncerModelBase objects parsed from `manifest.json`.\n        source (DbtBouncerSource): The DbtBouncerSourceBase object to check.\n\n    Other Parameters:\n        exclude (Optional[str]): Regex pattern to match the source path (i.e the .yml file where the source is configured). Source paths that match the pattern will not be checked.\n        include (Optional[str]): Regex pattern to match the source path (i.e the .yml file where the source is configured). Only source paths that match the pattern will be checked.\n        severity (Optional[Literal[\"error\", \"warn\"]]): Severity level of the check. Default: `error`.\n\n    Example(s):\n        ```yaml\n        manifest_checks:\n            - name: check_source_used_by_models_in_same_directory\n        ```\n\n    \"\"\"\n\n    models: List[\"DbtBouncerModelBase\"] = Field(default=[])\n    name: Literal[\"check_source_used_by_models_in_same_directory\"]\n    source: \"DbtBouncerSourceBase\" = Field(default=None)\n\n    def execute(self) -> None:\n        \"\"\"Execute the check.\"\"\"\n        reffed_models_not_in_same_dir = []\n        for model in self.models:\n            if (\n                self.source.unique_id in model.depends_on.nodes\n                and model.original_file_path.split(\"/\")[:-1]\n                != self.source.original_file_path.split(\"/\")[:-1]\n            ):\n                reffed_models_not_in_same_dir.append(model.unique_id.split(\".\")[0])\n\n        assert (\n            len(reffed_models_not_in_same_dir) == 0\n        ), f\"Source `{self.source.source_name}.{self.source.name}` is referenced by models defined in a different directory: {reffed_models_not_in_same_dir}\"\n
"},{"location":"checks/manifest/check_sources/#manifest.check_sources.CheckSourceUsedByOnlyOneModel","title":"CheckSourceUsedByOnlyOneModel","text":"

Each source can be referenced by a maximum of one model.

Receives at execution time:

Name Type Description models List[DbtBouncerModelBase]

List of DbtBouncerModelBase objects parsed from manifest.json.

source DbtBouncerSource

The DbtBouncerSourceBase object to check.

Other Parameters (passed via config file):

Name Type Description exclude Optional[str]

Regex pattern to match the source path (i.e the .yml file where the source is configured). Source paths that match the pattern will not be checked.

include Optional[str]

Regex pattern to match the source path (i.e the .yml file where the source is configured). Only source paths that match the pattern will be checked.

severity Optional[Literal['error', 'warn']]

Severity level of the check. Default: error.

Example(s):

manifest_checks:\n    - name: check_source_used_by_only_one_model\n

Source code in src/dbt_bouncer/checks/manifest/check_sources.py
class CheckSourceUsedByOnlyOneModel(BaseCheck):\n    \"\"\"Each source can be referenced by a maximum of one model.\n\n    Receives:\n        models (List[DbtBouncerModelBase]): List of DbtBouncerModelBase objects parsed from `manifest.json`.\n        source (DbtBouncerSource): The DbtBouncerSourceBase object to check.\n\n    Other Parameters:\n        exclude (Optional[str]): Regex pattern to match the source path (i.e the .yml file where the source is configured). Source paths that match the pattern will not be checked.\n        include (Optional[str]): Regex pattern to match the source path (i.e the .yml file where the source is configured). Only source paths that match the pattern will be checked.\n        severity (Optional[Literal[\"error\", \"warn\"]]): Severity level of the check. Default: `error`.\n\n    Example(s):\n        ```yaml\n        manifest_checks:\n            - name: check_source_used_by_only_one_model\n        ```\n\n    \"\"\"\n\n    models: List[\"DbtBouncerModelBase\"] = Field(default=[])\n    name: Literal[\"check_source_used_by_only_one_model\"]\n    source: \"DbtBouncerSourceBase\" = Field(default=None)\n\n    def execute(self) -> None:\n        \"\"\"Execute the check.\"\"\"\n        num_refs = sum(\n            self.source.unique_id in model.depends_on.nodes for model in self.models\n        )\n        assert (\n            num_refs <= 1\n        ), f\"Source `{self.source.source_name}.{self.source.name}` is referenced by more than one model.\"\n
"},{"location":"checks/manifest/check_unit_tests/","title":"Manifest Checks: Unit Tests","text":"

Note

The below checks require manifest.json to be present.

"},{"location":"checks/manifest/check_unit_tests/#manifest.check_unit_tests.CheckUnitTestExpectFormats","title":"CheckUnitTestExpectFormats","text":"

Unit tests can only use the specified formats.

Warning

This check is only supported for dbt 1.8.0 and above.

Parameters:

Name Type Description Default permitted_formats Optional[List[Literal['csv', 'dict', 'sql']]]

A list of formats that are allowed to be used for expect input in a unit test.

required

Receives at execution time:

Name Type Description manifest_obj DbtBouncerManifest

The DbtBouncerManifest object parsed from manifest.json.

unit_test UnitTests

The UnitTests object to check.

Other Parameters (passed via config file):

Name Type Description exclude Optional[str]

Regex pattern to match the unit test path (i.e the .yml file where the unit test is configured). Unit test paths that match the pattern will not be checked.

include Optional[str]

Regex pattern to match the unit test path (i.e the .yml file where the unit test is configured). Only unit test paths that match the pattern will be checked.

severity Optional[Literal['error', 'warn']]

Severity level of the check. Default: error.

Example(s):

manifest_checks:\n    - name: check_unit_test_expect_format\n      permitted_formats:\n        - csv\n

Source code in src/dbt_bouncer/checks/manifest/check_unit_tests.py
class CheckUnitTestExpectFormats(BaseCheck):\n    \"\"\"Unit tests can only use the specified formats.\n\n    !!! warning\n\n        This check is only supported for dbt 1.8.0 and above.\n\n    Parameters:\n        permitted_formats (Optional[List[Literal[\"csv\", \"dict\", \"sql\"]]]): A list of formats that are allowed to be used for `expect` input in a unit test.\n\n    Receives:\n        manifest_obj (DbtBouncerManifest): The DbtBouncerManifest object parsed from `manifest.json`.\n        unit_test (UnitTests): The UnitTests object to check.\n\n    Other Parameters:\n        exclude (Optional[str]): Regex pattern to match the unit test path (i.e the .yml file where the unit test is configured). Unit test paths that match the pattern will not be checked.\n        include (Optional[str]): Regex pattern to match the unit test path (i.e the .yml file where the unit test is configured). Only unit test paths that match the pattern will be checked.\n        severity (Optional[Literal[\"error\", \"warn\"]]): Severity level of the check. Default: `error`.\n\n    Example(s):\n        ```yaml\n        manifest_checks:\n            - name: check_unit_test_expect_format\n              permitted_formats:\n                - csv\n        ```\n\n    \"\"\"\n\n    manifest_obj: \"DbtBouncerManifest\" = Field(default=None)\n    name: Literal[\"check_unit_test_expect_format\"]\n    permitted_formats: List[Literal[\"csv\", \"dict\", \"sql\"]] = Field(\n        default=[\"csv\", \"dict\", \"sql\"],\n    )\n    unit_test: \"UnitTests\" = Field(default=None)\n\n    def execute(self) -> None:\n        \"\"\"Execute the check.\"\"\"\n        if (\n            semver.Version.parse(self.manifest_obj.manifest.metadata.dbt_version)\n            >= \"1.8.0\"\n        ):\n            assert (\n                self.unit_test.expect.format.value in self.permitted_formats\n            ), f\"Unit test `{self.unit_test.name}` has an `expect` format that is not permitted. Permitted formats are: {self.permitted_formats}.\"\n        else:\n            logging.warning(\n                \"The `check_unit_test_expect_format` check is only supported for dbt 1.8.0 and above.\",\n            )\n
"},{"location":"checks/manifest/check_unit_tests/#manifest.check_unit_tests.CheckUnitTestGivenFormats","title":"CheckUnitTestGivenFormats","text":"

Unit tests can only use the specified formats.

Warning

This check is only supported for dbt 1.8.0 and above.

Parameters:

Name Type Description Default permitted_formats Optional[List[Literal['csv', 'dict', 'sql']]]

A list of formats that are allowed to be used for expect input in a unit test.

required

Receives at execution time:

Name Type Description manifest_obj DbtBouncerManifest

The DbtBouncerManifest object parsed from manifest.json.

unit_test UnitTests

The UnitTests object to check.

Other Parameters (passed via config file):

Name Type Description exclude Optional[str]

Regex pattern to match the unit test path (i.e the .yml file where the unit test is configured). Unit test paths that match the pattern will not be checked.

include Optional[str]

Regex pattern to match the unit test path (i.e the .yml file where the unit test is configured). Only unit test paths that match the pattern will be checked.

severity Optional[Literal['error', 'warn']]

Severity level of the check. Default: error.

Example(s):

manifest_checks:\n    - name: check_unit_test_given_formats\n      permitted_formats:\n        - csv\n

Source code in src/dbt_bouncer/checks/manifest/check_unit_tests.py
class CheckUnitTestGivenFormats(BaseCheck):\n    \"\"\"Unit tests can only use the specified formats.\n\n    !!! warning\n\n        This check is only supported for dbt 1.8.0 and above.\n\n    Parameters:\n        permitted_formats (Optional[List[Literal[\"csv\", \"dict\", \"sql\"]]]): A list of formats that are allowed to be used for `expect` input in a unit test.\n\n    Receives:\n        manifest_obj (DbtBouncerManifest): The DbtBouncerManifest object parsed from `manifest.json`.\n        unit_test (UnitTests): The UnitTests object to check.\n\n    Other Parameters:\n        exclude (Optional[str]): Regex pattern to match the unit test path (i.e the .yml file where the unit test is configured). Unit test paths that match the pattern will not be checked.\n        include (Optional[str]): Regex pattern to match the unit test path (i.e the .yml file where the unit test is configured). Only unit test paths that match the pattern will be checked.\n        severity (Optional[Literal[\"error\", \"warn\"]]): Severity level of the check. Default: `error`.\n\n    Example(s):\n        ```yaml\n        manifest_checks:\n            - name: check_unit_test_given_formats\n              permitted_formats:\n                - csv\n        ```\n\n    \"\"\"\n\n    manifest_obj: \"DbtBouncerManifest\" = Field(default=None)\n    name: Literal[\"check_unit_test_given_formats\"]\n    permitted_formats: List[Literal[\"csv\", \"dict\", \"sql\"]] = Field(\n        default=[\"csv\", \"dict\", \"sql\"],\n    )\n    unit_test: \"UnitTests\" = Field(default=None)\n\n    def execute(self) -> None:\n        \"\"\"Execute the check.\"\"\"\n        if (\n            semver.Version.parse(self.manifest_obj.manifest.metadata.dbt_version)\n            >= \"1.8.0\"\n        ):\n            given_formats = [i.format.value for i in self.unit_test.given]\n            assert all(\n                e in self.permitted_formats for e in given_formats\n            ), f\"Unit test `{self.unit_test.name}` has given formats which are not permitted. Permitted formats are: {self.permitted_formats}.\"\n        else:\n            logging.warning(\n                \"The `check_unit_test_given_formats` check is only supported for dbt 1.8.0 and above.\",\n            )\n
"},{"location":"checks/run_results/check_run_results/","title":"Run Results Checks","text":"

Note

The below checks require both manifest.json and run_results.json to be present.

"},{"location":"checks/run_results/check_run_results/#run_results.check_run_results.CheckRunResultsMaxGigabytesBilled","title":"CheckRunResultsMaxGigabytesBilled","text":"

Each result can have a maximum number of gigabytes billed.

Note

Note that this check only works for the dbt-bigquery adapter.

Parameters:

Name Type Description Default max_gigabytes_billed float

The maximum number of gigabytes billed.

required run_result DbtBouncerRunResult

The DbtBouncerRunResult object to check.

required

Other Parameters (passed via config file):

Name Type Description exclude Optional[str]

Regex pattern to match the resource path. Resource paths that match the pattern will not be checked.

include Optional[str]

Regex pattern to match the resource path. Only resource paths that match the pattern will be checked.

severity Optional[Literal['error', 'warn']]

Severity level of the check. Default: error.

Example(s):

run_results_checks:\n    - name: check_run_results_max_gigabytes_billed\n    max_gigabytes_billed: 100\n

Source code in src/dbt_bouncer/checks/run_results/check_run_results.py
class CheckRunResultsMaxGigabytesBilled(BaseCheck):\n    \"\"\"Each result can have a maximum number of gigabytes billed.\n\n    !!! note\n\n        Note that this check only works for the `dbt-bigquery` adapter.\n\n    Parameters:\n        max_gigabytes_billed (float): The maximum number of gigabytes billed.\n        run_result (DbtBouncerRunResult): The DbtBouncerRunResult object to check.\n\n    Other Parameters:\n        exclude (Optional[str]): Regex pattern to match the resource path. Resource paths that match the pattern will not be checked.\n        include (Optional[str]): Regex pattern to match the resource path. Only resource paths that match the pattern will be checked.\n        severity (Optional[Literal[\"error\", \"warn\"]]): Severity level of the check. Default: `error`.\n\n    Raises: # noqa:DOC502\n        KeyError: If the `dbt-bigquery` adapter is not used.\n\n    Example(s):\n        ```yaml\n        run_results_checks:\n            - name: check_run_results_max_gigabytes_billed\n            max_gigabytes_billed: 100\n        ```\n\n    \"\"\"\n\n    max_gigabytes_billed: float\n    name: Literal[\"check_run_results_max_gigabytes_billed\"]\n    run_result: Optional[\"DbtBouncerRunResultBase\"] = Field(default=None)\n\n    def execute(self) -> None:\n        \"\"\"Execute the check.\"\"\"\n        try:\n            gigabytes_billed = self.run_result.adapter_response[\"bytes_billed\"] / (\n                1000**3\n            )\n        except KeyError as e:\n            raise RuntimeError(  # noqa: DOC501\n                \"`bytes_billed` not found in adapter response. Are you using the `dbt-bigquery` adapter?\",\n            ) from e\n\n        assert (\n            gigabytes_billed < self.max_gigabytes_billed\n        ), f\"`{self.run_result.unique_id.split('.')[-2]}` results in ({gigabytes_billed} billed bytes, this is greater than permitted ({self.max_gigabytes_billed}).\"\n
"},{"location":"checks/run_results/check_run_results/#run_results.check_run_results.CheckRunResultsMaxExecutionTime","title":"CheckRunResultsMaxExecutionTime","text":"

Each result can take a maximum duration (seconds).

Parameters:

Name Type Description Default max_execution_time_seconds float

The maximum execution time (seconds) allowed for a node.

required

Receives at execution time:

Name Type Description run_result DbtBouncerRunResult

The DbtBouncerRunResult object to check.

Other Parameters (passed via config file):

Name Type Description exclude Optional[str]

Regex pattern to match the resource path. Resource paths that match the pattern will not be checked.

include Optional[str]

Regex pattern to match the resource path. Only resource paths that match the pattern will be checked.

severity Optional[Literal['error', 'warn']]

Severity level of the check. Default: error.

Example(s):

run_results_checks:\n    - name: check_run_results_max_execution_time\n      max_execution_time_seconds: 60\n
run_results_checks:\n    - name: check_run_results_max_execution_time\n      include: ^models/staging # Not a good idea, here for demonstration purposes only\n      max_execution_time_seconds: 10\n

Source code in src/dbt_bouncer/checks/run_results/check_run_results.py
class CheckRunResultsMaxExecutionTime(BaseCheck):\n    \"\"\"Each result can take a maximum duration (seconds).\n\n    Parameters:\n        max_execution_time_seconds (float): The maximum execution time (seconds) allowed for a node.\n\n    Receives:\n        run_result (DbtBouncerRunResult): The DbtBouncerRunResult object to check.\n\n    Other Parameters:\n        exclude (Optional[str]): Regex pattern to match the resource path. Resource paths that match the pattern will not be checked.\n        include (Optional[str]): Regex pattern to match the resource path. Only resource paths that match the pattern will be checked.\n        severity (Optional[Literal[\"error\", \"warn\"]]): Severity level of the check. Default: `error`.\n\n    Example(s):\n        ```yaml\n        run_results_checks:\n            - name: check_run_results_max_execution_time\n              max_execution_time_seconds: 60\n        ```\n        ```yaml\n        run_results_checks:\n            - name: check_run_results_max_execution_time\n              include: ^models/staging # Not a good idea, here for demonstration purposes only\n              max_execution_time_seconds: 10\n        ```\n\n    \"\"\"\n\n    max_execution_time_seconds: float\n    name: Literal[\"check_run_results_max_execution_time\"]\n    run_result: Optional[\"DbtBouncerRunResultBase\"] = Field(default=None)\n\n    def execute(self) -> None:\n        \"\"\"Execute the check.\"\"\"\n        assert (\n            self.run_result.execution_time <= self.max_execution_time_seconds\n        ), f\"`{self.run_result.unique_id.split('.')[-1]}` has an execution time ({self.run_result.execution_time} greater than permitted ({self.max_execution_time_seconds}s).\"\n
"}]} \ No newline at end of file