Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

v1.13.2 - Update Flyte components #5791

Merged
merged 16 commits into from
Oct 2, 2024
Merged

v1.13.2 - Update Flyte components #5791

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
16 commits
Select commit Hold shift + click to select a range
8537609
Update Flyte Components
eapolinario Oct 1, 2024
d9cd83e
Add changelog and bump version in `docs/conf.py`
eapolinario Oct 1, 2024
be1f957
Fix typo in changelog
eapolinario Oct 1, 2024
928ffd0
Install plotly in single-binary.yml
eapolinario Oct 1, 2024
c4b780c
Install other dependencies in single-binary.yml
eapolinario Oct 1, 2024
94e62a3
Debug - use flytekit 1.13.5
eapolinario Oct 1, 2024
a2ca0d5
Revert "Debug - use flytekit 1.13.5"
eapolinario Oct 1, 2024
e28ee79
Debug - -vv
eapolinario Oct 1, 2024
b6f0151
Debug - run debug version
eapolinario Oct 1, 2024
85b4a16
Use change from https://github.com/flyteorg/flytekit/pull/2782
eapolinario Oct 2, 2024
ded4dde
Merge remote-tracking branch 'origin' into flyte-bot-update-releases
eapolinario Oct 2, 2024
12ebbfe
Remove debug statements
eapolinario Oct 2, 2024
25a0263
Add pandas to single-binary tests
eapolinario Oct 2, 2024
1bc2041
Put flytekit_version back
eapolinario Oct 2, 2024
3e34481
Update changelog
eapolinario Oct 2, 2024
22f1365
Fix typo in changelog
eapolinario Oct 2, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions .github/workflows/single-binary.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -175,7 +175,7 @@ jobs:
run: |
python -m pip install --upgrade pip
pip install uv
uv pip install --system flytekit==1.13.5 flytekitplugins-deck-standard==1.13.5 "numpy<2.0.0" pyarrow
uv pip install --system flytekit flytekitplugins-deck-standard "numpy<2.0.0" pyarrow pandas
uv pip freeze
- name: Checkout flytesnacks
uses: actions/checkout@v4
Expand All @@ -188,7 +188,7 @@ jobs:
flytekit_version=$(pip show flytekit | grep -i version | awk '{ print $2 }')
while read -r line;
do
pyflyte --config ./boilerplate/flyte/end2end/functional-test-config.yaml \
pyflyte -vv --config ./boilerplate/flyte/end2end/functional-test-config.yaml \
register \
--project flytesnacks \
--domain development \
Expand Down
53 changes: 53 additions & 0 deletions CHANGELOG/CHANGELOG-v1.13.2.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,53 @@
# Flyte 1.13.2 Release Notes

## What's Changed
* Enable echo plugin by default by @pingsutw in https://github.com/flyteorg/flyte/pull/5679
* Do not emit execution id label by default in single binary by @eapolinario in https://github.com/flyteorg/flyte/pull/5704
* Using new offloaded metadata literal message for literal offloading by @pmahindrakar-oss in https://github.com/flyteorg/flyte/pull/5705
* Improve error message for MismatchingTypes by @pingsutw in https://github.com/flyteorg/flyte/pull/5639
* [Docs] Echo Task by @Future-Outlier in https://github.com/flyteorg/flyte/pull/5707
* Improve execution name readability by @wayner0628 in https://github.com/flyteorg/flyte/pull/5637
* Configure imagePullPolicy to be Always pull on flyte sandbox environment by @pmahindrakar-oss in https://github.com/flyteorg/flyte/pull/5709
* Set IsDefault to False for echo plugin by @pingsutw in https://github.com/flyteorg/flyte/pull/5713
* Move default execution name generation to flyteadmin by @wayner0628 in https://github.com/flyteorg/flyte/pull/5714
* Update helm/docs per changes in supported task discovery by @Sovietaced in https://github.com/flyteorg/flyte/pull/5694
* [flyteagent] Add Logging for Agent Supported Task Types by @Future-Outlier in https://github.com/flyteorg/flyte/pull/5718
* extend pod customization to include init containers by @samhita-alla in https://github.com/flyteorg/flyte/pull/5685
* Update "Try Serverless" language in Quickstart guide by @neverett in https://github.com/flyteorg/flyte/pull/5698
* Refactor flyteadmin to pass proto structs as pointers by @Sovietaced in https://github.com/flyteorg/flyte/pull/5717
* fix: Use deterministic execution names in scheduler by @pingsutw in https://github.com/flyteorg/flyte/pull/5724
* [flyteagent] Enable `agent-service` in every configuration by @Future-Outlier in https://github.com/flyteorg/flyte/pull/5730
* [Docs][flyteagent] Remove `agent-service` config in flyte agent documentation by @Future-Outlier in https://github.com/flyteorg/flyte/pull/5731
* Fix flytectl returning the oldest workflow when using --latest flag by @RRap0so in https://github.com/flyteorg/flyte/pull/5716
* Remove explicit go toolchain versions by @ddl-ebrown in https://github.com/flyteorg/flyte/pull/5723
* Add listing api to stow storage by @bgedik in https://github.com/flyteorg/flyte/pull/5741
* Use latest upload/download-artifact action version by @Sovietaced in https://github.com/flyteorg/flyte/pull/5743
* Introduced SMTP notification by @robert-ulbrich-mercedes-benz in https://github.com/flyteorg/flyte/pull/5535
* Added literal offloading for array node map tasks by @pmahindrakar-oss in https://github.com/flyteorg/flyte/pull/5697
* pytorch object.inv moved by @wild-endeavor in https://github.com/flyteorg/flyte/pull/5755
* [RFC] Binary IDL With MessagePack Bytes by @Future-Outlier in https://github.com/flyteorg/flyte/pull/5742
* Rename literal offloading flags and enable offloading in local config by @eapolinario in https://github.com/flyteorg/flyte/pull/5754
* Add semver check for dev and beta version by @pmahindrakar-oss in https://github.com/flyteorg/flyte/pull/5757
* Add comments for semver regex by @pmahindrakar-oss in https://github.com/flyteorg/flyte/pull/5758
* Enable caching in flyte-core helmchart by @cpaulik in https://github.com/flyteorg/flyte/pull/5739
* [Flyte][1][IDL] Binary IDL With MessagePack by @Future-Outlier in https://github.com/flyteorg/flyte/pull/5751
* [Flyte][2][Literal Type For Scalar] Binary IDL With MessagePack by @Future-Outlier in https://github.com/flyteorg/flyte/pull/5761
* Backoff on etcd errors by @EngHabu in https://github.com/flyteorg/flyte/pull/5710
* [Docs][flyteagent] Update Databricks Agent Setup to V2 by @Future-Outlier in https://github.com/flyteorg/flyte/pull/5766
* [Docs][flyteagent] Improve Agent Secret Setup by @Future-Outlier in https://github.com/flyteorg/flyte/pull/5765
* [RFC] Offloaded Raw Literals by @wild-endeavor in https://github.com/flyteorg/flyte/pull/5103
* Use pluggable clock for auto refresh cache and make unit tests fast by @Sovietaced in https://github.com/flyteorg/flyte/pull/5767
* [Docs] Flyte Deck example by @Future-Outlier in https://github.com/flyteorg/flyte/pull/5777
* Update ImageSpec documentation by @pingsutw in https://github.com/flyteorg/flyte/pull/5748
* Revert "Improve execution name readability" by @pingsutw in https://github.com/flyteorg/flyte/pull/5740
* [Docs] Flyte Deck example V2 by @Future-Outlier in https://github.com/flyteorg/flyte/pull/5781
* [flytepropeller][flyteadmin] Compiler unknown literal type error handling by @Future-Outlier in https://github.com/flyteorg/flyte/pull/5651
* [Temporary fix] Pin flytekit version for single binary builds by @pmahindrakar-oss in https://github.com/flyteorg/flyte/pull/5798
* Fix propeller crash when inferring literal type for an offloaded literal by @pmahindrakar-oss in https://github.com/flyteorg/flyte/pull/5771

## New Contributors
* @wayner0628 made their first contribution in https://github.com/flyteorg/flyte/pull/5637
* @robert-ulbrich-mercedes-benz made their first contribution in https://github.com/flyteorg/flyte/pull/5535
* @cpaulik made their first contribution in https://github.com/flyteorg/flyte/pull/5739

**Full Changelog**: https://github.com/flyteorg/flyte/compare/v1.13.1...v1.13.2
2 changes: 1 addition & 1 deletion charts/flyte-binary/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,7 @@ Chart for basic single Flyte executable deployment
| configuration.auth.oidc.clientId | string | `""` | |
| configuration.auth.oidc.clientSecret | string | `""` | |
| configuration.co-pilot.image.repository | string | `"cr.flyte.org/flyteorg/flytecopilot"` | |
| configuration.co-pilot.image.tag | string | `"v1.13.1"` | |
| configuration.co-pilot.image.tag | string | `"v1.13.2"` | |
| configuration.database.dbname | string | `"flyte"` | |
| configuration.database.host | string | `"127.0.0.1"` | |
| configuration.database.options | string | `"sslmode=disable"` | |
Expand Down
2 changes: 1 addition & 1 deletion charts/flyte-binary/values.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -159,7 +159,7 @@ configuration:
# repository CoPilot sidecar image repository
repository: cr.flyte.org/flyteorg/flytecopilot # FLYTECOPILOT_IMAGE
# tag CoPilot sidecar image tag
tag: v1.13.1 # FLYTECOPILOT_TAG
tag: v1.13.2 # FLYTECOPILOT_TAG
# agentService Flyte Agent configuration
agentService:
defaultAgent:
Expand Down
12 changes: 6 additions & 6 deletions charts/flyte-core/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -95,8 +95,8 @@ helm install gateway bitnami/contour -n flyte
| configmap.clusters.clusterConfigs | list | `[]` | |
| configmap.clusters.labelClusterMap | object | `{}` | |
| configmap.console | object | `{"BASE_URL":"/console","CONFIG_DIR":"/etc/flyte/config"}` | Configuration for Flyte console UI |
| configmap.copilot | object | `{"plugins":{"k8s":{"co-pilot":{"image":"cr.flyte.org/flyteorg/flytecopilot:v1.13.1","name":"flyte-copilot-","start-timeout":"30s"}}}}` | Copilot configuration |
| configmap.copilot.plugins.k8s.co-pilot | object | `{"image":"cr.flyte.org/flyteorg/flytecopilot:v1.13.1","name":"flyte-copilot-","start-timeout":"30s"}` | Structure documented [here](https://pkg.go.dev/github.com/lyft/[email protected]/go/tasks/pluginmachinery/flytek8s/config#FlyteCoPilotConfig) |
| configmap.copilot | object | `{"plugins":{"k8s":{"co-pilot":{"image":"cr.flyte.org/flyteorg/flytecopilot:v1.13.2","name":"flyte-copilot-","start-timeout":"30s"}}}}` | Copilot configuration |
| configmap.copilot.plugins.k8s.co-pilot | object | `{"image":"cr.flyte.org/flyteorg/flytecopilot:v1.13.2","name":"flyte-copilot-","start-timeout":"30s"}` | Structure documented [here](https://pkg.go.dev/github.com/lyft/[email protected]/go/tasks/pluginmachinery/flytek8s/config#FlyteCoPilotConfig) |
| configmap.core | object | `{"manager":{"pod-application":"flytepropeller","pod-template-container-name":"flytepropeller","pod-template-name":"flytepropeller-template"},"propeller":{"downstream-eval-duration":"30s","enable-admin-launcher":true,"leader-election":{"enabled":true,"lease-duration":"15s","lock-config-map":{"name":"propeller-leader","namespace":"flyte"},"renew-deadline":"10s","retry-period":"2s"},"limit-namespace":"all","max-workflow-retries":30,"metadata-prefix":"metadata/propeller","metrics-prefix":"flyte","prof-port":10254,"queue":{"batch-size":-1,"batching-interval":"2s","queue":{"base-delay":"5s","capacity":1000,"max-delay":"120s","rate":100,"type":"maxof"},"sub-queue":{"capacity":100,"rate":10,"type":"bucket"},"type":"batch"},"rawoutput-prefix":"s3://my-s3-bucket/","workers":4,"workflow-reeval-duration":"30s"},"webhook":{"certDir":"/etc/webhook/certs","serviceName":"flyte-pod-webhook"}}` | Core propeller configuration |
| configmap.core.manager | object | `{"pod-application":"flytepropeller","pod-template-container-name":"flytepropeller","pod-template-name":"flytepropeller-template"}` | follows the structure specified [here](https://pkg.go.dev/github.com/flyteorg/flytepropeller/manager/config#Config). |
| configmap.core.propeller | object | `{"downstream-eval-duration":"30s","enable-admin-launcher":true,"leader-election":{"enabled":true,"lease-duration":"15s","lock-config-map":{"name":"propeller-leader","namespace":"flyte"},"renew-deadline":"10s","retry-period":"2s"},"limit-namespace":"all","max-workflow-retries":30,"metadata-prefix":"metadata/propeller","metrics-prefix":"flyte","prof-port":10254,"queue":{"batch-size":-1,"batching-interval":"2s","queue":{"base-delay":"5s","capacity":1000,"max-delay":"120s","rate":100,"type":"maxof"},"sub-queue":{"capacity":100,"rate":10,"type":"bucket"},"type":"batch"},"rawoutput-prefix":"s3://my-s3-bucket/","workers":4,"workflow-reeval-duration":"30s"}` | follows the structure specified [here](https://pkg.go.dev/github.com/flyteorg/flytepropeller/pkg/controller/config). |
Expand Down Expand Up @@ -130,7 +130,7 @@ helm install gateway bitnami/contour -n flyte
| datacatalog.extraArgs | object | `{}` | Appends extra command line arguments to the main command |
| datacatalog.image.pullPolicy | string | `"IfNotPresent"` | Docker image pull policy |
| datacatalog.image.repository | string | `"cr.flyte.org/flyteorg/datacatalog"` | Docker image for Datacatalog deployment |
| datacatalog.image.tag | string | `"v1.13.1"` | Docker image tag |
| datacatalog.image.tag | string | `"v1.13.2"` | Docker image tag |
| datacatalog.nodeSelector | object | `{}` | nodeSelector for Datacatalog deployment |
| datacatalog.podAnnotations | object | `{}` | Annotations for Datacatalog pods |
| datacatalog.podEnv | object | `{}` | Additional Datacatalog container environment variables |
Expand Down Expand Up @@ -166,7 +166,7 @@ helm install gateway bitnami/contour -n flyte
| flyteadmin.extraArgs | object | `{}` | Appends extra command line arguments to the serve command |
| flyteadmin.image.pullPolicy | string | `"IfNotPresent"` | |
| flyteadmin.image.repository | string | `"cr.flyte.org/flyteorg/flyteadmin"` | Docker image for Flyteadmin deployment |
| flyteadmin.image.tag | string | `"v1.13.1"` | |
| flyteadmin.image.tag | string | `"v1.13.2"` | |
| flyteadmin.initialProjects | list | `["flytesnacks","flytetester","flyteexamples"]` | Initial projects to create |
| flyteadmin.nodeSelector | object | `{}` | nodeSelector for Flyteadmin deployment |
| flyteadmin.podAnnotations | object | `{}` | Annotations for Flyteadmin pods |
Expand Down Expand Up @@ -238,7 +238,7 @@ helm install gateway bitnami/contour -n flyte
| flytepropeller.extraArgs | object | `{}` | Appends extra command line arguments to the main command |
| flytepropeller.image.pullPolicy | string | `"IfNotPresent"` | |
| flytepropeller.image.repository | string | `"cr.flyte.org/flyteorg/flytepropeller"` | Docker image for Flytepropeller deployment |
| flytepropeller.image.tag | string | `"v1.13.1"` | |
| flytepropeller.image.tag | string | `"v1.13.2"` | |
| flytepropeller.manager | bool | `false` | |
| flytepropeller.nodeSelector | object | `{}` | nodeSelector for Flytepropeller deployment |
| flytepropeller.podAnnotations | object | `{}` | Annotations for Flytepropeller pods |
Expand Down Expand Up @@ -270,7 +270,7 @@ helm install gateway bitnami/contour -n flyte
| flytescheduler.configPath | string | `"/etc/flyte/config/*.yaml"` | Default regex string for searching configuration files |
| flytescheduler.image.pullPolicy | string | `"IfNotPresent"` | Docker image pull policy |
| flytescheduler.image.repository | string | `"cr.flyte.org/flyteorg/flytescheduler"` | Docker image for Flytescheduler deployment |
| flytescheduler.image.tag | string | `"v1.13.1"` | Docker image tag |
| flytescheduler.image.tag | string | `"v1.13.2"` | Docker image tag |
| flytescheduler.nodeSelector | object | `{}` | nodeSelector for Flytescheduler deployment |
| flytescheduler.podAnnotations | object | `{}` | Annotations for Flytescheduler pods |
| flytescheduler.podEnv | object | `{}` | Additional Flytescheduler container environment variables |
Expand Down
10 changes: 5 additions & 5 deletions charts/flyte-core/values.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ flyteadmin:
image:
# -- Docker image for Flyteadmin deployment
repository: cr.flyte.org/flyteorg/flyteadmin # FLYTEADMIN_IMAGE
tag: v1.13.1 # FLYTEADMIN_TAG
tag: v1.13.2 # FLYTEADMIN_TAG
pullPolicy: IfNotPresent
# -- Additional flyteadmin container environment variables
#
Expand Down Expand Up @@ -144,7 +144,7 @@ flytescheduler:
# -- Docker image for Flytescheduler deployment
repository: cr.flyte.org/flyteorg/flytescheduler # FLYTESCHEDULER_IMAGE
# -- Docker image tag
tag: v1.13.1 # FLYTESCHEDULER_TAG
tag: v1.13.2 # FLYTESCHEDULER_TAG
# -- Docker image pull policy
pullPolicy: IfNotPresent
# -- Default resources requests and limits for Flytescheduler deployment
Expand Down Expand Up @@ -210,7 +210,7 @@ datacatalog:
# -- Docker image for Datacatalog deployment
repository: cr.flyte.org/flyteorg/datacatalog # DATACATALOG_IMAGE
# -- Docker image tag
tag: v1.13.1 # DATACATALOG_TAG
tag: v1.13.2 # DATACATALOG_TAG
# -- Docker image pull policy
pullPolicy: IfNotPresent
# -- Default resources requests and limits for Datacatalog deployment
Expand Down Expand Up @@ -307,7 +307,7 @@ flytepropeller:
image:
# -- Docker image for Flytepropeller deployment
repository: cr.flyte.org/flyteorg/flytepropeller # FLYTEPROPELLER_IMAGE
tag: v1.13.1 # FLYTEPROPELLER_TAG
tag: v1.13.2 # FLYTEPROPELLER_TAG
pullPolicy: IfNotPresent
# -- Default resources requests and limits for Flytepropeller deployment
resources:
Expand Down Expand Up @@ -799,7 +799,7 @@ configmap:
# -- Structure documented [here](https://pkg.go.dev/github.com/lyft/[email protected]/go/tasks/pluginmachinery/flytek8s/config#FlyteCoPilotConfig)
co-pilot:
name: flyte-copilot-
image: cr.flyte.org/flyteorg/flytecopilot:v1.13.1 # FLYTECOPILOT_IMAGE
image: cr.flyte.org/flyteorg/flytecopilot:v1.13.2 # FLYTECOPILOT_IMAGE
start-timeout: 30s

# -- Core propeller configuration
Expand Down
Loading
Loading