-
Notifications
You must be signed in to change notification settings - Fork 662
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Updates to deployment guides #3994
Changes from all commits
db159e1
4063745
d36e9dc
08692a7
be2abfd
4c51dc2
70737b7
bb25ab6
cf77de9
56cfed8
828359b
c7f7e3e
18f0fc8
e5cea21
e9b685b
b257695
ee5c001
7db1b7f
0e2d871
9199f41
db07bab
83d8c35
e784efd
fb0bb64
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -49,29 +49,6 @@ deployment comes with a containerized `Minio <https://min.io/>`__, which offers | |
- **GCP**: `GCS <https://cloud.google.com/storage/>`__ | ||
- **Azure**: `Azure Blob Storage <https://azure.microsoft.com/en-us/products/storage/blobs>`__ | ||
|
||
|
||
Cluster Configuration | ||
===================== | ||
|
||
Flyte configures K8s clusters to work with it. For example, as your Flyte userbase evolves, adding new projects is as | ||
simple as registering them through the command line: | ||
|
||
.. prompt:: bash $ | ||
|
||
flytectl create project \ | ||
--id my-flyte-project \ | ||
--name "My Flyte Project" \ | ||
--description "My first project onboarding onto Flyte" | ||
|
||
Once you invoke this command, this project should immediately show up in the Flyte console after refreshing. | ||
|
||
Flyte runs at a configurable cadence that ensures that all Kubernetes resources necessary for the new project are | ||
created and new workflows can successfully be registered and executed within it. | ||
|
||
.. note:: | ||
|
||
For more information, see :std:ref:`flytectl <flytectl:flytectl_create_project>`. | ||
|
||
************************ | ||
Flyte Deployment Paths | ||
************************ | ||
|
@@ -108,7 +85,10 @@ There are three different paths for deploying a Flyte cluster: | |
This option is appropriate if all your compute can `fit on one EKS cluster <https://docs.aws.amazon.com/eks/latest/userguide/service-quotas.html>`__ . | ||
As of this writing, a single Flyte cluster can handle more than 13,000 nodes. | ||
|
||
Whatever path you choose, note that ``FlytePropeller`` itself can be sharded as well, though typically it's not required. | ||
Regardless of using single or multiple Kubernetes clusters for Flyte, note that ``FlytePropeller`` -the main data plane component- can be scaled out as well by using ``sharding`` if scale demands require it. | ||
See `Automatic scale-out <https://docs.flyte.org/en/latest/deployment/configuration/performance.html#automatic-scale-out>`__ to learn more about the sharding mechanism. | ||
|
||
|
||
|
||
Helm | ||
==== | ||
|
@@ -156,10 +136,13 @@ Deployment Tips and Tricks | |
|
||
Due to the many choices and constraints that you may face in your organization, the specific steps for deploying Flyte | ||
can vary significantly. For example, which cloud platform to use is typically a big fork in the road for many, and there | ||
are many choices to make in terms of ingresses, auth providers, and versions of different dependent libraries that | ||
are many choices to make in terms of Ingress controllers, auth providers, and versions of different dependent libraries that | ||
may interact with other parts of your stack. | ||
|
||
In addition to searching and posting on the `Flyte Slack community <https://flyte-org.slack.com/archives/C01P3B761A6>`__, | ||
Considering the above, we recommend checking out the `"Flyte The Hard Way" <https://github.com/davidmirror-ops/flyte-the-hard-way/tree/main#flyte-the-hard-way>`__ set of community-maintained tutorials that can guide you through the process of preparing the infrastructure and | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. In general I would avoid pointing to an external guide, give also that the guide you are referring to is mostly referring to the single binary deployment, which might be confusing. I think all documentation should be in one place There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Agree. A second iteration of updates to these guides should be pointed to extend the reach and make it more actionable. Also expanding the tutorial in the https://github.com/unionai-oss/deploy-flyte repo to cover There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. what if we forked your guide, @davidmirror-ops , and moved it to flyteorg? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. @eapolinario I was thinking of something similar. Moving forward, the idea is that TF is the preferred approach but a manual guide is always a good resource |
||
deploying Flyte. | ||
|
||
In addition to searching and posting on the `#flyte-deployment Slack channel <https://flyte-org.slack.com/archives/C01P3B761A6>`__, | ||
we have a `Github Discussion <https://github.com/flyteorg/flyte/discussions/categories/deployment-tips-tricks>`__ | ||
section dedicated to deploying Flyte. Feel free to submit any hints you've found helpful as a discussion, ask questions, | ||
or simply document what worked or what didn't work for you. | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why are we removing this section?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm just not sure how it fits here. Additionally is not clear what specific K8s resources it refers to (besides namespaces for projects, which is only an example). So this relationship between Flyte objects and K8s resources is worth documenting, but I'm not sure it fits in this sort of pre-requisites section.