Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Adaptive Job] Model freshness spec validation #11124

Open
1 task done
ChenyuLInx opened this issue Dec 11, 2024 · 0 comments
Open
1 task done

[Adaptive Job] Model freshness spec validation #11124

ChenyuLInx opened this issue Dec 11, 2024 · 0 comments
Labels
user docs [docs.getdbt.com] Needs better documentation

Comments

@ChenyuLInx
Copy link
Contributor

Housekeeping

  • I am a maintainer of dbt-core

Short description

User should not define models that have build after period models smaller than all models it depends on. dbt will raise Error during parsing if the follow happens:

example DAG: model 1 -> model 3, model 2 -> model 3

  • model 1: 2h, model 2: 2h, model 3: 1 h
  • model 1: 1h, model 2: 2h, model 3: 1 h, depends_on: all

For transitory dependencies:
example DAG: model 1 -> model 3, model 2 -> model 3, model 3 -> model 4,

  • if model 1: 1h, model 2: 2h, model 3: depends_on: all, then model 3's implied frequency is 2h
  • if model 1: 1h, model 2: 2h, model 3: depends_on: any, then model 3's implied frequency is 1h

Acceptance criteria

Proper error is raised during parsing time.

Suggested Tests

Various valid and invalid model definitons with proper raise/not raise checked.

Impact to Other Teams

No

Will backports be required?

No

Context

No response

@ChenyuLInx ChenyuLInx added the user docs [docs.getdbt.com] Needs better documentation label Dec 11, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
user docs [docs.getdbt.com] Needs better documentation
Projects
None yet
Development

No branches or pull requests

1 participant