Skip to content

Commit

Permalink
Add docs
Browse files Browse the repository at this point in the history
  • Loading branch information
pankajastro committed Dec 19, 2024
1 parent 8ba587d commit a417dec
Show file tree
Hide file tree
Showing 4 changed files with 18 additions and 2 deletions.
3 changes: 2 additions & 1 deletion cosmos/airflow/graph.py
Original file line number Diff line number Diff line change
Expand Up @@ -177,7 +177,8 @@ def create_task_metadata(
:param dbt_dag_task_group_identifier: Identifier to refer to the DbtDAG or DbtTaskGroup in the DAG.
:param use_task_group: It determines whether to use the name as a prefix for the task id or not.
If it is False, then use the name as a prefix for the task id, otherwise do not.
:param on_warning_callback:
:param on_warning_callback: A callback function called on warnings with additional Context variables “test_names”
and “test_results” of type List. This is param available for dbt test and dbt source freshness command.
:returns: The metadata necessary to instantiate the source dbt node as an Airflow task.
"""
dbt_resource_to_class = create_dbt_resource_to_class(test_behavior)
Expand Down
1 change: 0 additions & 1 deletion cosmos/core/airflow.py
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,6 @@ def get_airflow_task(task: Task, dag: DAG, task_group: TaskGroup | None = None)
for k, v in task.airflow_task_config.items():
task_kwargs[k] = v

# Set the on_warning_callback of source node in task_kwargs
on_warning_callback = task.extra_context.get("on_warning_callback")
if on_warning_callback:
task_kwargs["on_warning_callback"] = on_warning_callback
Expand Down
3 changes: 3 additions & 0 deletions dev/dags/example_source_rendering.py
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,8 @@
),
)

# [START cosmos_source_node_example]

source_rendering_dag = DbtDag(
# dbt/cosmos-specific parameters
project_config=ProjectConfig(
Expand All @@ -42,3 +44,4 @@
default_args={"retries": 2},
on_warning_callback=lambda context: print(context),
)
# [END cosmos_source_node_example]
13 changes: 13 additions & 0 deletions docs/configuration/source-nodes-rendering.rst
Original file line number Diff line number Diff line change
Expand Up @@ -34,3 +34,16 @@ Example:
source_rendering_behavior=SourceRenderingBehavior.WITH_TESTS_OR_FRESHNESS,
)
)
on_warning_callback Callback
~~~~~~~~~~~~~~~~~~~~~~~~~~~~

The ``on_warning_callback`` is a callback parameter available on the ``DbtSourceLocalOperator``. This callback is triggered when a warning occurs during the execution of the ``dbt source freshness`` command. The callback accepts the task context, which includes additional parameters: test_names and test_results

Example:

.. literalinclude:: ../../dev/dags/example_source_rendering.py/
:language: python
:start-after: [START cosmos_source_node_example]
:end-before: [END cosmos_source_node_example]

0 comments on commit a417dec

Please sign in to comment.