Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Fix CI issue running integration tests (#640)
At the moment, all PRs are failing due to issues with integration tests not being able to run. An example of error message: ``` FAILED tests/test_example_dags.py::test_example_dag[extract_dag] - assert not {'/home/runner/work/astronomer-cosmos/astronomer-cosmos/dev/dags/example_cosmos_python_models.py': 'Traceback (most recent call last):\n File "/opt/hostedtoolcache/Python/3.9.18/x64/lib/python3.9/selectors.py", line 416, in select\n fd_event_list = self._selector.poll(timeout)\n File "/home/runner/.local/share/hatch/env/virtual/astronomer-cosmos/Za_bFbg4/tests.py3.9-2.4/lib/python3.9/site-packages/airflow/utils/timeout.py", line 69, in handle_timeout\n raise AirflowTaskTimeout(self.error_message)\nairflow.exceptions.AirflowTaskTimeout: DagBag import timeout for /home/runner/work/astronomer-cosmos/astronomer-cosmos/dev/dags/example_cosmos_python_models.py after 30.0s.\nPlease take a look at these docs to improve your DAG import time:\n* https://airflow.apache.org/docs/apache-airflow/2.4.0/best-practices.html#top-level-python-code\n* https://airflow.apache.org/docs/apache-airflow/2.4.0/best-practices.html#reducing-dag-complexity, PID: 2558\n'} ``` This issue relates to #520 - something we'll be soon working on. In the meantime, we're increasing the DAG Parsing time in the CI - so our tests can run again. We should also review how we run the integration tests - we probably do not need to run all DAGs against the Python x Airflow matrix, and we could parallelise some of those.
- Loading branch information