Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add cycle detection info #4398

Merged
merged 5 commits into from
Nov 3, 2023
Merged

add cycle detection info #4398

Changes from 3 commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
abaedc2
add cycle detection info
mirnawong1 Nov 3, 2023
58a5628
Merge branch 'current' into mirnawong1-patch-21
mirnawong1 Nov 3, 2023
2007ba3
Update website/docs/guides/best-practices/how-we-mesh/mesh-2-structur…
mirnawong1 Nov 3, 2023
2fa141e
Update website/docs/guides/best-practices/how-we-mesh/mesh-2-structur…
mirnawong1 Nov 3, 2023
0e74ad0
Update website/docs/guides/best-practices/how-we-mesh/mesh-2-structur…
mirnawong1 Nov 3, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions website/docs/guides/best-practices/how-we-mesh/mesh-2-structures.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,10 @@ At a high level, you’ll need to decide:
- Where to draw the lines between your dbt Projects -- i.e. how do you determine where to split your DAG and which models go in which project?
- How to manage your code -- do you want multiple dbt Projects living in the same repository (mono-repo) or do you want to have multiple repos with one repo per project?

### Cycle detection

Currently, project dependencies only work in a one-way direction. This helps you avoid indefinite project `ref` cycles (or loops) and issues with your data workflows. For example, the `jaffle_finance"` project can't introduce a new model that depends on `jaffle_marketing.roi_by_channel`. Refer to [Project dependencies](/docs/collaborate/govern/project-dependencies#how-to-use-ref) for more information.
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Currently, project dependencies only work in a one-way direction. This helps you avoid indefinite project `ref` cycles (or loops) and issues with your data workflows. For example, the `jaffle_finance"` project can't introduce a new model that depends on `jaffle_marketing.roi_by_channel`. Refer to [Project dependencies](/docs/collaborate/govern/project-dependencies#how-to-use-ref) for more information.
Currently, project dependencies only work in a one-way direction. This helps you avoid indefinite project `ref` cycles (or loops) and also avoid issues with your data workflows. For example, the `jaffle_finance"` project can't introduce a new model that depends on `jaffle_marketing.roi_by_channel`. Refer to [Project dependencies](/docs/collaborate/govern/project-dependencies#how-to-use-ref) for more information.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

adding avoid twice because i feel the 2nd issue gets lost. might be overthinking it

mirnawong1 marked this conversation as resolved.
Show resolved Hide resolved
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see a stray " here:

jaffle_finance"

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thank you good catch! i think @matthewshaver caught it too 🙏

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think since we don't mention the jaffle projects until page 3, we shouldn't bring them up here!

Suggested change
Currently, project dependencies only work in a one-way direction. This helps you avoid indefinite project `ref` cycles (or loops) and issues with your data workflows. For example, the `jaffle_finance"` project can't introduce a new model that depends on `jaffle_marketing.roi_by_channel`. Refer to [Project dependencies](/docs/collaborate/govern/project-dependencies#how-to-use-ref) for more information.
Like resource dependencies, project dependencies are acyclic, meaning they only move in one direction. This prevents `ref` cycles (or loops), which lead to issues with your data workflows. For example, if project B depends on project A, a new model in project A could not import and use a public model from project B. Refer to [Project dependencies](/docs/collaborate/govern/project-dependencies#how-to-use-ref) for more information.


## Define your project interfaces by splitting your DAG

The first (and perhaps most difficult!) decision when migrating to a multi-project architecture is deciding where to draw the line in your DAG to define the interfaces between your projects. Let's explore some language for discussing the design of these patterns.
Expand Down
Loading