Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[CT-1580] [Bug] Warning message produces parameter value from another macro argument, not the one configured #6355

Closed
2 tasks done
kd7vrc opened this issue Dec 1, 2022 · 3 comments
Labels
bug Something isn't working user docs [docs.getdbt.com] Needs better documentation wontfix Not a bug or out of scope for dbt-core

Comments

@kd7vrc
Copy link

kd7vrc commented Dec 1, 2022

Is this a new bug in dbt-core?

  • I believe this is a new bug in dbt-core
  • I have searched the existing issues, and I could not find an existing issue for this bug

Current Behavior

A macro that accepts two arguments, and warns based on a condition, and contains one of the parameters in the message outputs the other parameter value in the compile log. (Correct at execution.)

Expected Behavior

A parameterized warning message should produce the correct values for the corresponding parameter.

Steps To Reproduce

  1. Create a macro fitting the description like this:
{% macro exception_warn_bug(source_rel, target_rel) %}
{# Defaults for easy test #}
{% set source_rel = source_rel or ref('wk_umb_policy_trig_pc') %}{# Exists #}
{% set target_rel = target_rel or source('etl_policy_dbt_tgt','cv_umb_insured_subject_pc') %}{# Does not exist #}
{% set allow_target_not_exists = allow_target_not_exists or True %}
{{ log("Begin processing from " ~ source_rel ~ " to " ~ target_rel, True) }}
{%- if load_relation(target_rel) is none -%}
    {% do exceptions.warn("Target '" ~ target_rel ~ "' does not exist on the database.", True) %}
{% elif execute %}
    {% do log('Execute stuff', True) %}
{% endif %}
{% endmacro %}```
2. Add it somewhere so dbt IDE compiles it `{{ exception_warn_bug }}`
3. Review the **compilation** logs

### Relevant log output

```shell
# From compile log
18:56:42  Begin processing from RRADDEV.PLZ4614.wk_umb_policy_trig_pc to RRADDEV.PLZ4614.wk_umb_policy_trig_pc
18:56:42  Target 'RRADDEV.PLZ4614.wk_umb_policy_trig_pc' does not exist on the database.

Environment

- dbt Cloud: 1.3

Which database adapter are you using with dbt?

snowflake

Additional Context

Documentation indicates that the warning is intended to be used for compilation warning. See https://docs.getdbt.com/reference/dbt-jinja-functions/exceptions#warn

@kd7vrc kd7vrc added bug Something isn't working triage labels Dec 1, 2022
@github-actions github-actions bot changed the title [Bug] Warning message produces parameter value from another macro argument, not the one configured [CT-1580] [Bug] Warning message produces parameter value from another macro argument, not the one configured Dec 1, 2022
@dbeatty10 dbeatty10 self-assigned this Dec 2, 2022
@jtcohen6
Copy link
Contributor

jtcohen6 commented Dec 2, 2022

(Sorry @dbeatty10 , this one happened to catch my eye!)

At parse time:

  • load_relation is always going to return None, because the cache has not yet been populated.
  • ref() and source() return a placeholder value, which actually point to the current model's relation — because we don't know about how to resolve those other project objects yet! We're still parsing. See ParseRefResolver and ParseSourceResolver. The idea is to still return some sort of Relation, rather than None, because the latter would break code expecting to operate on a Relation-type object (as it will be doing at compile/runtime).

This has confused several other folks: #2793 (comment), plus several issues linked to/from there. I think we need to document it! Maybe here? https://docs.getdbt.com/reference/dbt-jinja-functions/execute

Another confusing piece of this might be that we're logging / warning at all during parsing. Macros like log() and exceptions.warn() are still evaluated at parse time, during our "first-pass" Jinja render to extract ref/source/config. This is misleading. Related: #5360

Of course, it's possible to avoid this with a modification to the code above:

{%- if execute and load_relation(target_rel) is none -%}

@dbeatty10 dbeatty10 added user docs [docs.getdbt.com] Needs better documentation Team:Language labels Dec 2, 2022
@dbeatty10 dbeatty10 added wontfix Not a bug or out of scope for dbt-core and removed triage labels Dec 5, 2022
@dbeatty10
Copy link
Contributor

@kd7vrc Thanks again for reaching out about this!

We're going to address this with some updates to the docs and close this as wontfix. But we'd welcome your thoughts on what a better behavior might look like, though.

Here's the issue we opened as a result:
dbt-labs/docs.getdbt.com#2492

We'd value your feedback on those docs updates as well.

@dbeatty10 dbeatty10 closed this as not planned Won't fix, can't repro, duplicate, stale Dec 5, 2022
@dbeatty10 dbeatty10 removed their assignment Dec 5, 2022
@elyobo
Copy link

elyobo commented Feb 3, 2023

@dbeatty10 Does wontfix imply that you're not fixing #2793 as well? That's not just a "causes confusion" issue, that's a "prevents very useful $$$ saving and performance improving functionality in BigQuery" issue - specifically it would be very nice to be able to accurately reference models in the SQL header.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working user docs [docs.getdbt.com] Needs better documentation wontfix Not a bug or out of scope for dbt-core
Projects
None yet
Development

No branches or pull requests

4 participants