diff --git a/rsts/deployment/agents/bigquery.rst b/rsts/deployment/agents/bigquery.rst new file mode 100644 index 0000000000..ca2b704986 --- /dev/null +++ b/rsts/deployment/agents/bigquery.rst @@ -0,0 +1,90 @@ +.. _deployment-agent-setup-bigquery: + +Google BigQuery Agent +====================== + +This guide provides an overview of setting up BigQuery agent in your Flyte deployment. +Please note that the BigQuery agent requires Flyte deployment in the GCP cloud; +it is not compatible with demo/AWS/Azure. + +Set up the GCP Flyte cluster +---------------------------- + +* Ensure you have a functional Flyte cluster running in `GCP `__. +* Create a service account for BigQuery. For more details, refer to: https://cloud.google.com/bigquery/docs/quickstarts/quickstart-client-libraries. +* Verify that you have the correct kubeconfig and have selected the appropriate Kubernetes context. +* Confirm that you have the correct Flytectl configuration at ``~/.flyte/config.yaml``. + +Specify agent configuration +---------------------------- + +.. tabs:: + + .. group-tab:: Flyte binary + + Edit the relevant YAML file to specify the agent. + + .. code-block:: yaml + :emphasize-lines: 7,11 + + tasks: + task-plugins: + enabled-plugins: + - container + - sidecar + - k8s-array + - bigquery + default-for-task-types: + - container: container + - container_array: k8s-array + - bigquery_query_job_task: agent-service + + .. group-tab:: Flyte core + + Create a file named ``values-override.yaml`` and add the following configuration to it. + + .. code-block:: yaml + + configmap: + enabled_plugins: + # -- Tasks specific configuration [structure](https://pkg.go.dev/github.com/flyteorg/flytepropeller/pkg/controller/nodes/task/config#GetConfig) + tasks: + # -- Plugins configuration, [structure](https://pkg.go.dev/github.com/flyteorg/flytepropeller/pkg/controller/nodes/task/config#TaskPluginConfig) + task-plugins: + # -- [Enabled Plugins](https://pkg.go.dev/github.com/flyteorg/flyteplugins/go/tasks/config#Config). Enable sagemaker*, athena if you install the backend + enabled-plugins: + - container + - sidecar + - k8s-array + - bigquery + default-for-task-types: + container: container + sidecar: sidecar + container_array: k8s-array + bigquery_query_job_task: agent-service + +Ensure that the propeller has the correct service account for BigQuery. + +Upgrade the Flyte Helm release +------------------------------ + +.. tabs:: + + .. group-tab:: Flyte binary + + .. code-block:: bash + + helm upgrade flyteorg/flyte-binary -n --values + + Replace ```` with the name of your release (e.g., ``flyte-backend``), + ```` with the name of your namespace (e.g., ``flyte``), + and ```` with the name of your YAML file. + + .. group-tab:: Flyte core + + .. code-block:: bash + + helm upgrade flyte/flyte-core -n --values values-override.yaml + + Replace ```` with the name of your release (e.g., ``flyte``) + and ```` with the name of your namespace (e.g., ``flyte``). diff --git a/rsts/deployment/agents/index.rst b/rsts/deployment/agents/index.rst new file mode 100644 index 0000000000..57232864a7 --- /dev/null +++ b/rsts/deployment/agents/index.rst @@ -0,0 +1,26 @@ +.. _deployment-agent-setup: + +Agent Setup +=========== + +.. tags:: Agent, Integration, Data, Advanced + +Discover the process of setting up Agents for Flyte. + +.. panels:: + :header: text-center + :column: col-lg-12 p-2 + + .. link-button:: deployment-agent-setup-bigquery + :type: ref + :text: Bigquery Agent + :classes: btn-block stretched-link + ^^^^^^^^^^^^ + Guide to setting up the Bigquery agent. + +.. toctree:: + :maxdepth: 1 + :name: Agent setup + :hidden: + + bigquery diff --git a/rsts/deployment/index.rst b/rsts/deployment/index.rst index 9417c28204..8f569eb843 100644 --- a/rsts/deployment/index.rst +++ b/rsts/deployment/index.rst @@ -31,6 +31,15 @@ plugins, authentication, performance tuning, and maintaining Flyte as a producti --- + .. link-button:: deployment-agent-setup + :type: ref + :text: 🤖 Agent Setup + :classes: btn-block stretched-link + ^^^^^^^^^^^^ + Enable Flyte agents to extend Flyte's capabilities, including features like File sesnor, Databricks job, and Snowflake query services. + + --- + .. link-button:: deployment-configuration :type: ref :text: 🎛 Cluster Configuration diff --git a/rsts/index.rst b/rsts/index.rst index 6ea3176607..9468f92cf5 100644 --- a/rsts/index.rst +++ b/rsts/index.rst @@ -31,6 +31,7 @@ deployment/index deployment/deployment/index deployment/plugins/index + deployment/agents/index deployment/configuration/index deployment/configuration/generated/index deployment/security/index