-
Notifications
You must be signed in to change notification settings - Fork 26
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Deprecate EmrContainerOperatorAsync #1402
Conversation
* feat(amazon): deprecate EmrContainerOperatorAsync * feat(amazon): remove EmrContainerOperatorTrigger
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## main #1402 +/- ##
==========================================
- Coverage 98.54% 98.41% -0.13%
==========================================
Files 91 91
Lines 5364 5312 -52
==========================================
- Hits 5286 5228 -58
- Misses 78 84 +6 ☔ View full report in Codecov by Sentry. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
self.log.info(event["message"]) | ||
job_id: str = event["job_id"] | ||
return job_id | ||
return super().__init__(*args, deferrable=True, **kwargs) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Do we need a return here?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Nope, i think it's accidentally added. let me send a quick fix
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What's changed
Deprecate
EmrContainerOperatorAsync
and fallback toEmrContainerOperator
withdeferrable=True
This is blocked by the release of
apache/airflow#36417apache/airflow#36467upgrade minimum apache-airflow-providers-amazon to 8.15.0
Why this change
The logic of
EmrContainerOperatorAsync
has been contributed back to OSS airflow as part ofEmrContainerOperator
EmrContainerOperatorAsync
astronomer-providers/astronomer/providers/amazon/aws/sensors/emr.py
Lines 38 to 65 in b5a9417
EmrContainerOperator
https://github.com/apache/airflow/blob/999b70178a1f5d891fd2c88af4831a4ba4c2cbc9/airflow/providers/amazon/aws/sensors/emr.py#L318-L346