[Feature] Add callback support for ExecutionMode.KUBERNETES

[pankajkoti]

Description

Similar to the functionality of supporting to specify callback argument in the ExecutionMode.LOCAL as can be seen in

astronomer-cosmos/cosmos/operators/local.py

Line 143 in 0000f80

callback: Callable[[str], None] | None = None,

, and as PR #1389 demonstrates how to leverage this callback argument, we should add the support for the callback argument in ExceutionMode.KUBERNETES

Use case/motivation

A common request from users is the ability to inspect files in the target directory after task execution. Since Cosmos clones the project directory into a temporary directory and executes tasks from there, access to this directory is lost once the task completes. While ExecutionMode.LOCAL provides callback support to perform actions on this temporary directory, similar functionality is currently unavailable for ExecutionMode.Kubernetes. Adding this support would be highly beneficial.

Related issues

related: #1349

Are you willing to submit a PR?

• Yes, I am willing to submit a PR!