[Bug] dbt --warn-error test results in compilation error when tests are not yet renamed on dbt-core>=1.8.0

[fivetran-joemarkiewicz]

Is this a new bug in dbt-core?

• I believe this is a new bug in dbt-core
• I have searched the existing issues, and I could not find an existing issue for this bug

Current Behavior

I wasn't sure if this was a regression or a new bug 🤔. Let me know if you would like me to make any edits if you believe this is a regression.

When running dbt --warn-error test immediately after running dbt clean && dbt deps and the tests have not yet been renamed to data_tests we see a compilation failure.

Please note that this failure does not occur when you execute dbt run or dbt build on a cleaned project. This only happens with the dbt --warn-error test command.

Expected Behavior

The dbt --warn-error test will still work as expected for the time being while the tests naming convention is still allowed. I expect an error like this once the tests naming convention is fully removed.

Steps To Reproduce

1. dbt clean
2. dbt deps
3. dbt --warn-error test
4. See compilation error

Relevant log output

16:59:30  Running with dbt=1.8.1
16:59:31  Registered adapter: bigquery=1.8.1
16:59:31  Unable to do partial parsing because saved manifest not found. Starting full parse.
16:59:32  Encountered an error:
Compilation Error
  [WARNING]: Deprecated functionality
  The `tests` config has been renamed to `data_tests`. Please see
  https://docs.getdbt.com/docs/build/data-tests#new-data_tests-syntax for more
  information.

Environment

- OS: MacOS
- Python: 3.10.11
- dbt: 1.8.1

Which database adapter are you using with dbt?

bigquery

Additional Context

I was able to recreate this compilation error using the above steps, but this also came out of a bug raised on the Fivetran Google Ads package which seemed odd as I would not expect this warning to cause the users entire tests framework to not be able to be executed.

[dbeatty10]

Thanks for opening this @fivetran-joemarkiewicz !

TLDR

Try this instead:

dbt --warn-error-options '{"error": "all", "silence": ["TestsConfigDeprecation"]}' test

warn_error vs. warn_error_options

This documentation explains what is going on when including the --warn-error flag:

Turning on the WARN_ERROR config will convert dbt warnings into errors. Any time dbt would normally warn, it will instead raise an error. Examples include --select criteria that selects no resources, deprecations, configurations with no associated models, invalid test configurations, or tests and freshness checks that are configured to return warnings.

The WARN_ERROR_OPTIONS config is a mutually exclusive alternative to WARN_ERROR that allows specifying exactly which types of warnings are treated as errors.

It's is a lot more verbose, but it gives a lot more control.

For example, these two are equivalent:

dbt --warn-error test
dbt --warn-error-options '{"error": "all"}' test

Using warn_error_options to silence warnings

CLI flag

By switching to the latter option, it makes it possible to silence the deprecation warning for the tests config being renamed to data_tests:

dbt --warn-error-options '{"error": "all", "silence": ["TestsConfigDeprecation"]}' test

Environment variable

Alternatively, it's possible to set WARN_ERROR_OPTIONS via environment variable:

export DBT_WARN_ERROR_OPTIONS='{"error": "all", "silence": ["TestsConfigDeprecation"]}'
dbt test

Project flag

Lastly, it can be set in dbt_project.yml as well*:

flags:
  warn_error_options:
    error: all
    silence:
      - TestsConfigDeprecation

* This example in dbt_project.yml isn't working for me at the moment, so I'm going to open a separate issue see #10160.

I don't think the issue you reported is a bug though, so I'm going to close this as "not planned". I may have missed something important -- if so, just let me know 🙏

[fivetran-joemarkiewicz]

Thanks @dbeatty10 for taking a look and providing the guidance! 🙏 I will share your suggestions with the OP from the issue they raised in the Fivetran Google Ads package.