Update apache/airflow Docker tag to v2.10.2 #85
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
![renovate[bot]](https://avatars.githubusercontent.com/in/2740?s=60&v=4){kind=small}
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR contains the following updates:
2.9.2->2.10.2Release Notes
apache/airflow (apache/airflow)
v2.10.2: Apache Airflow 2.10.2Compare Source
Significant Changes
No significant changes.
Bug Fixes
renderedTemplatesas keys to skipcamelCasing(#42206) (#42208)camelcasexcom entries (#42182) (#42187)Miscellaneous
0.2.4as it breaks our integration (#42101)LibCST(#42089)--treeflag fortasks listcli command (#41965)Doc Only Changes
security_model.rstto clear unauthenticated endpoints exceptions (#42085)v2.10.1Compare Source
Significant Changes
^^^^^^^^^^^^^^^^^^^
No significant changes.
Bug Fixes
"""""""""
tojsonfilter to example_inlet_event_extra example dag (#41890)Miscellaneous
"""""""""""""
Doc Only Changes
""""""""""""""""
keycloak(#41791)v2.10.0Compare Source
Significant Changes
^^^^^^^^^^^^^^^^^^^
Datasets no longer trigger inactive DAGs (#38891)
"""""""""""""""""""""""""""""""""""""""""""""""""
Previously, when a DAG is paused or removed, incoming dataset events would still
trigger it, and the DAG would run when it is unpaused or added back in a DAG
file. This has been changed; a DAG's dataset schedule can now only be satisfied
by events that occur when the DAG is active. While this is a breaking change,
the previous behavior is considered a bug.
The behavior of time-based scheduling is unchanged, including the timetable part
of
DatasetOrTimeSchedule.try_numberis no longer incremented during task execution (#39336)""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""
Previously, the try number (
try_number) was incremented at the beginning of task execution on the worker. This was problematic for many reasons.For one it meant that the try number was incremented when it was not supposed to, namely when resuming from reschedule or deferral. And it also resulted in
the try number being "wrong" when the task had not yet started. The workarounds for these two issues caused a lot of confusion.
Now, instead, the try number for a task run is determined at the time the task is scheduled, and does not change in flight, and it is never decremented.
So after the task runs, the observed try number remains the same as it was when the task was running; only when there is a "new try" will the try number be incremented again.
One consequence of this change is, if users were "manually" running tasks (e.g. by calling
ti.run()directly, or command lineairflow tasks run),try number will no longer be incremented. Airflow assumes that tasks are always run after being scheduled by the scheduler, so we do not regard this as a breaking change.
/logoutendpoint in FAB Auth Manager is now CSRF protected (#40145)"""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""
The
/logoutendpoint's method in FAB Auth Manager has been changed fromGETtoPOSTin all existingAuthViews (
AuthDBView,AuthLDAPView,AuthOAuthView,AuthOIDView,AuthRemoteUserView), andnow includes CSRF protection to enhance security and prevent unauthorized logouts.
OpenTelemetry Traces for Apache Airflow (#37948).
"""""""""""""""""""""""""""""""""""""""""""""""""
This new feature adds capability for Apache Airflow to emit 1) airflow system traces of scheduler,
triggerer, executor, processor 2) DAG run traces for deployed DAG runs in OpenTelemetry format. Previously, only metrics were supported which emitted metrics in OpenTelemetry.
This new feature will add richer data for users to use OpenTelemetry standard to emit and send their trace data to OTLP compatible endpoints.
Decorator for Task Flow
(@​skip_if, @​run_if)to make it simple to apply whether or not to skip a Task. (#41116)""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""
This feature adds a decorator to make it simple to skip a Task.
Using Multiple Executors Concurrently (#40701)
""""""""""""""""""""""""""""""""""""""""""""""
Previously known as hybrid executors, this new feature allows Airflow to use multiple executors concurrently. DAGs, or even individual tasks, can be configured
to use a specific executor that suits its needs best. A single DAG can contain tasks all using different executors. Please see the Airflow documentation for
more details. Note: This feature is still experimental. See
documentation on Executor <https://airflow.apache.org/docs/apache-airflow/stable/core-concepts/executor/index.html#using-multiple-executors-concurrently>_ for a more detailed description.Scarf based telemetry: Does Airflow collect any telemetry data? (#39510)
""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""
Airflow integrates Scarf to collect basic usage data during operation. Deployments can opt-out of data collection by setting the
[usage_data_collection]enabledoption to False, or the SCARF_ANALYTICS=false environment variable.See
FAQ on this <https://airflow.apache.org/docs/apache-airflow/stable/faq.html#does-airflow-collect-any-telemetry-data>_ for more information.New Features
""""""""""""
AIP-61 <https://github.com/apache/airflow/pulls?q=is%3Apr+label%3Aarea%3Ahybrid-executors+is%3Aclosed+milestone%3A%22Airflow+2.10.0%22>_)AIP-62 <https://github.com/apache/airflow/pulls?q=is%3Apr+is%3Amerged+label%3AAIP-62+milestone%3A%22Airflow+2.10.0%22>_)AIP-64 <https://github.com/apache/airflow/pulls?q=is%3Apr+is%3Amerged+label%3AAIP-64+milestone%3A%22Airflow+2.10.0%22>_)AIP-44 <https://github.com/apache/airflow/pulls?q=is%3Apr+label%3AAIP-44+milestone%3A%22Airflow+2.10.0%22+is%3Aclosed>_)accessorsto read dataset events defined as inlet (#39367)dag test(#40010)endDatein task instance tooltip. (#39547)accessorsto read dataset events defined as inlet (#39367, #39893)run_if&skip_ifdecorators (#41116)Improvements
""""""""""""
renderedjsoncomponent (#40964)get_extra_dejsonmethod with nested parameter which allows you to specify if you want the nested json as string to be also deserialized (#39811)__getattr__to task decorator stub (#39425)RemovedIn20Warninginairflow taskcommand (#39244)db migrateerror messages (#39268)suppress_and_warnwarning (#39263)declarative_basefromsqlalchemy.orminstead ofsqlalchemy.ext.declarative(#39134)on_task_instance_failedaccess to the error that caused the failure (#38155)output_processorparameter toBashProcessor(#40843)Bug Fixes
"""""""""
never_failin BaseSensor (#40915)start_date(#40878)external_task_group_idtoWorkflowTrigger(#39617)BaseSensorOperatorintroduceskip_policyparameter (#40924)__init__(#41086)Miscellaneous
"""""""""""""
OTelTraces (#40874)pydocstylerules to pyproject.toml (#40569)pydocstylerule D213 in ruff. (#40448, #40464)Dag.test()to run with an executor if desired (#40205)AirflowInternalRuntimeErrorfor raisenon catchableerrors (#38778)pytestto 8.0+ (#39450)back_populatesbetweenDagScheduleDatasetReference.dagandDagModel.schedule_dataset_references(#39392)B028(no-explicit-stacklevel) in core (#39123)ImportErrortoParseImportErrorfor avoid shadowing with builtin exception (#39116)SubDagOperatorexamples warnings (#39057)model_dumpinstead ofdictfor serialize Pydantic V2 model (#38933)wsfrom 7.5.5 to 7.5.10 in /airflow/www (#40288)Doc Only Changes
""""""""""""""""
filesystemsanddataset-uristo "how to create your own provider" page (#40801)otel_onto True in example airflow.cfg (#40712)task_idfromsend_emailtosend_email_notificationintaskflow.rst(#41060)v2.9.3Compare Source
Significant Changes
^^^^^^^^^^^^^^^^^^^
Time unit for
scheduled_durationandqueued_durationchanged (#37936)"""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""
scheduled_durationandqueued_durationmetrics are now emitted in milliseconds instead of seconds.By convention all statsd metrics should be emitted in milliseconds, this is later expected in e.g.
prometheusstatsd-exporter.Support for OpenTelemetry Metrics is no longer "Experimental" (#40286)
""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""
Experimental support for OpenTelemetry was added in 2.7.0 since then fixes and improvements were added and now we announce the feature as stable.
Bug Fixes
"""""""""
[webserver]update_fab_permsto deprecated configs (#40317)httpxtorequestsinfile_task_handler(#39799)Doc Only Changes
""""""""""""""""
Miscellaneous
"""""""""""""
SchedulerJobRunner._process_executor_events(#40563)Configuration
📅 Schedule: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined).
🚦 Automerge: Disabled by config. Please merge this manually once you are satisfied.
♻ Rebasing: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.
🔕 Ignore: Close this PR and you won't be reminded about this update again.
This PR was generated by Mend Renovate. View the repository job log.