Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Fix dag randering for taskflow + DbtTaskGroup combo (#1360)
## Description I believe the deepcopy results in two separate DAG objects—the original object and its deepcopy and the task is referencing to both objects causing failure ``` airflow.exceptions.AirflowException: Tried to set relationships between tasks in more than one DAG: {<DAG: DAG_NAME>, <DAG: DAG_NAME>} ``` <img width="1681" alt="Screenshot 2024-12-03 at 2 30 23 AM" src="https://github.com/user-attachments/assets/93859f4b-84d1-4e60-87a7-25a1b750a8e3"> DAG Code ```python from datetime import datetime from airflow.decorators import task, dag from cosmos import DbtTaskGroup, ProjectConfig from include.constants import jaffle_shop_path, venv_execution_config, manifest_path from include.profiles import airflow_db @task(task_id="build_partial_dbt_env_vars_operator") def build_partial_dbt_env(): # some code # This return is for demonstration purposes only return { "ENV_VAR_NAME": "value", "ENV_VAR_NAME_2": False } # partial_dbt_env = build_partial_dbt_env() @dag( schedule_interval="@daily", start_date=datetime(2023, 1, 1), catchup=False, tags=["simple"], ) def simple_task_group1() -> None: DbtTaskGroup( group_id="transform_task_group", project_config=ProjectConfig( dbt_project_path=jaffle_shop_path, manifest_path=manifest_path, env_vars=build_partial_dbt_env() ), profile_config=airflow_db, execution_config=venv_execution_config, ) simple_task_group1() #partial_dbt_env >> transform_task_group ``` <!-- Add a brief but complete description of the change. --> ## Related Issue(s) closes: #1218 ## Breaking Change? <!-- If this introduces a breaking change, specify that here. --> ## Checklist - [ ] I have made corresponding changes to the documentation (if required) - [ ] I have added tests that prove my fix is effective or that my feature works
- Loading branch information