raise MicrobatchModelNoEventTimeInputs warning when no microbatch input has event_time config #10929
Conversation
…ut has event_time config
|
Thank you for your pull request! We could not find a changelog entry for this change. For details on how to document a change, see the contributing guide. |
MichelleArk
changed the title
core/dbt/events/types.py
Outdated
| def message(self) -> str: | ||
| msg = ( | ||
| f"The microbatch model '{self.model_name}' has no 'ref' or 'source' input with an 'event_time' configuration. " | ||
| "This can result in unexpected duplicate records in the resulting microbatch model." |
There was a problem hiding this comment.
cc @graciegoheen for any input on the warning message
There was a problem hiding this comment.
This can result in unexpected duplicate records in the resulting microbatch model.
Thoughts on this vs. "This means no filtering can be applied and can result in unexpected duplicate records in the resulting microbatch model."
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #10929 +/- ##
==========================================
- Coverage 89.18% 87.75% -1.43%
==========================================
Files 183 183
Lines 23476 23491 +15
==========================================
- Hits 20936 20615 -321
- Misses 2540 2876 +336
Flags with carried forward coverage won't be shown. Click here to find out more.
|
MichelleArk
deleted the
microbatch-warn-when-no-event-time-config-input
branch
Resolves #10926
Problem
It can be unintuitive for users that they need to configure microbatch models by adding
event_timefields to input models. For microbatch models with noevent_timeconfiguration, this can lead to unexpected duplicate data given it is likely not the intended use case to have microbatch but no event time filters.Solution
Emit a warning only when no inputs to a microbatch model have an event_time config.
Checklist
🎩
