We read every piece of feedback, and take your input very seriously.
To see all available qualifiers, see our documentation.
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Is there any way to run two tasks (with subtasks) in parallel, but setting some dependencies between the subtasks?
A concrete example:
Requirements:
Using Airflow (the only other orchestrator I have used), this would be done in the following way:
A1 >> A2 >> A3 B1 >> B2 >> B3 A2 >> B3
Is there any way to replicate this functionality using DigDag's declarative syntax?
Thank you.
EDIT: I have thought about two options,
option 1: doesn't exactly work because it makes a3 also wait for b2:
+task1: _parallel: true +a: +a1: ... +a2: ... +b: +b1: ... +b2: ... +task2: _parallel: true +a: +a3: ... +b: +b3: ...
option 2: doesn't exactly work because then b3 also has to wait for a3
+task1: _parallel: true +a: +a1: ... +a2: ... +a3: ... +b: +b1: ... +b2: ... +task2: +b: +b3: ...
The text was updated successfully, but these errors were encountered:
No branches or pull requests
Is there any way to run two tasks (with subtasks) in parallel, but setting some dependencies between the subtasks?
A concrete example:
Requirements:
Using Airflow (the only other orchestrator I have used), this would be done in the following way:
Is there any way to replicate this functionality using DigDag's declarative syntax?
Thank you.
EDIT: I have thought about two options,
option 1: doesn't exactly work because it makes a3 also wait for b2:
option 2: doesn't exactly work because then b3 also has to wait for a3
The text was updated successfully, but these errors were encountered: