Validate dbt resource identifier to see if it is a valid airflow task id during conversion from dbt to Airflow #657
Labels
area:parsing
Related to parsing DAG/DBT improvement, issues, or fixes
area:rendering
Related to rendering, like Jinja, Airflow tasks, etc
enhancement
New feature or request
Milestone
While parsing the dbt project, we should confirm if the node names we'll use to define Airflow task IDs are valid.
If they are not valid, we should at least raise a warning (probably an error message) explaining the issue to users.
We can accomplish this by using
airflow.utils.helpers.validate_key
.This task is related to #636, but they are not duplicates and should not conflict.
This improvement came from a conversation with @MrBones757 , regarding Cosmos 1.2.2. More details:
When we create dbtNode objects in https://github.com/astronomer/astronomer-cosmos/blob/main/cosmos/dbt/graph.py#L367
We set the nodeName == alias, with a fallback to name if it doesn't exist.
We then create the airflow graph with that value here: https://github.com/astronomer/astronomer-cosmos/blob/main/cosmos/airflow/graph.py#L125
The issue: dbt doesn't impose limitations on node names, but airflow does as such. When we go to render the airflow dag in the UI, we get:
The text was updated successfully, but these errors were encountered: