[CT-3233] [Bug] var() cannot see vars defined in dbt_project.yml in query-comment context

[slotrans]

Is this a new bug in dbt-core?

• I believe this is a new bug in dbt-core
• I have searched the existing issues, and I could not find an existing issue for this bug

Current Behavior

According to the docs var() is available in the Jinja context for query-comment. However, it only partially works.

When calling var() in the value of query-comment, either as a templated literal or in a macro, vars passed on the DBT command line using --var work as expected. But vars defined in dbt_project.yml are not found.

Judging by the error message it seems as if vars specified in dbt_project.yml are unavailable when query-comment is being evaluated. I cannot find any documentation to that effect (in particular neither here nor here).

Expected Behavior

I expect that var() is able to resolve the values of vars defined in dbt_project.yml, in all cases, including in the context of query-comment.

Steps To Reproduce

dbt_project.yml:

name: 'dbt_query_comment_test'
version: '1.0.0'
config-version: 2
profile: 'default'
# all the standard paths...
models:
  dbt_query_comment_test:
    +materialized: table
vars:
  foo: "bar"
query-comment: "foo is {{ var('foo') }}"

result of dbt run:

01:22:37  Running with dbt=1.6.6
01:22:37  Registered adapter: postgres=1.6.6
01:22:37  Unable to do partial parsing because a project config has changed
01:22:38  Encountered an error:
Compilation Error
  Required var 'foo' not found in config:
  Vars supplied to <Configuration> = {}

Similarly, if we write a macro like what's in the docs:

{% macro get_query_comment(node) %}
    {% set comment_dict = {"some_key": "some_value"} %}

    {% set _foo = var("foo") %}
    {% do log("_foo=" ~ _foo, info=True) %}
    {% do comment_dict.update(foo=_foo) %}

    {% do return(tojson(comment_dict)) %}
{% endmacro %}

...and adjust dbt_project.yml:

query-comment: "{{ get_query_comment(node) }}"

...and do dbt run:

01:25:21  Running with dbt=1.6.6
01:25:21  Registered adapter: postgres=1.6.6
01:25:22  Encountered an error:
Compilation Error
  Required var 'foo' not found in config:
  Vars supplied to <Configuration> = {}

  > in macro get_query_comment (macros/get_query_comment.sql)
  > called by <Unknown>

Note that running dbt run --vars '{foo: bar}' does work, with both a simple templated query comment, and with the macro call.

(I normally use DBT with BigQuery but I put together this reproduction case with Postgres. I have no reason to believe this behavior is adapter-specific.)

Relevant log output

No response

Environment

- OS: MacOS 12.5.1 (Monterey)
- Python: 3.10.6
- dbt: 1.6.6

Which database adapter are you using with dbt?

postgres, bigquery

Additional Context

No response

[dbeatty10]

Thanks for an excellent write-up @slotrans!

Documentation

Judging by the error message it seems as if vars specified in dbt_project.yml are unavailable when query-comment is being evaluated. I cannot find any documentation to that effect (in particular neither here nor here).

Indeed, within the dbt_project.yml context, only variables defined with --vars are available when using var. This is called out here, but we should consider adding it to one or both of the pages that you mentioned at well (and maybe here too).

Feature request

Regardless of the documentation, it still isn't doing what you hope or expect!

Namely it doesn't allow you to define vars within the dbt_project.yml context and then use them within your query-comments configuration 💥

👉 I think #2955 would accomplish exactly what you are looking for. We would appreciate any comments or upvotes you want to add to that one. In the meantime, I added a quick comment here that notes your use-case.

I'm going to close this as covered by #2955. But I've opened dbt-labs/docs.getdbt.com#4301 to improve the documentation about the current behavior.