Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Flyte Agent Helm Chart #3935

Merged
merged 22 commits into from
Sep 1, 2023
Merged
Show file tree
Hide file tree
Changes from 15 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/helm-charts.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ jobs:
strategy:
fail-fast: false
matrix:
chart: ["flyte-binary", "flyte-core"]
chart: ["flyte-binary", "flyte-core", "flyte-agent"]
steps:
- name: Checkout
uses: actions/checkout@v3
Expand Down
2 changes: 2 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,8 @@ __pycache__/
/charts/flyte/charts
/charts/flyte-deps/charts
/charts/flyte/Chart.lock
/charts/flyte-binary/Chart.lock
/charts/flyte-agent/Chart.lock
/charts/flyte-deps/Chart.lock
/charts/flyte-core/Chart.lock
/docker/sandbox-lite/storage/charts
Expand Down
23 changes: 23 additions & 0 deletions charts/flyte-agent/.helmignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
# Patterns to ignore when building packages.
# This supports shell glob matching, relative path matching, and
# negation (prefixed with !). Only one pattern per line.
.DS_Store
# Common VCS dirs
.git/
.gitignore
.bzr/
.bzrignore
.hg/
.hgignore
.svn/
# Common backup files
*.swp
*.bak
*.tmp
*.orig
*~
# Various IDEs
.project
.idea/
*.tmproj
.vscode/
5 changes: 5 additions & 0 deletions charts/flyte-agent/Chart.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
apiVersion: v2
name: flyteagent
description: A Helm chart for Flyte agent
type: application
version: v0.1.10 # VERSION
39 changes: 39 additions & 0 deletions charts/flyte-agent/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
# flyteagent

![Version: v0.1.10](https://img.shields.io/badge/Version-v0.1.10-informational?style=flat-square) ![Type: application](https://img.shields.io/badge/Type-application-informational?style=flat-square)

A Helm chart for Flyte agent

## Values

| Key | Type | Default | Description |
|-----|------|---------|-------------|
| additionalContainers | list | `[]` | Appends additional containers to the deployment spec. May include template values. |
| additionalVolumeMounts | list | `[]` | Appends additional volume mounts to the main container's spec. May include template values. |
| additionalVolumes | list | `[]` | Appends additional volumes to the deployment spec. May include template values. |
| affinity | object | `{}` | affinity for flyteagent deployment |
| agentSecret.name | string | `""` | Specify name of K8s Secret. Leave it empty if you don't need this Secret |
| agentSecret.secretData | object | `{"data":{"username":"User"}}` | Specify your Secret (with sensitive data) or pseudo-manifest (without sensitive data). See https://github.com/godaddy/kubernetes-external-secrets |
Copy link
Contributor

@jeevb jeevb Aug 9, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the updated link: https://github.com/external-secrets/external-secrets. For my own understanding, how can external-secrets be used with this block?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sorry, forgot to remove it

| commonAnnotations | object | `{}` | |
| commonLabels | object | `{}` | |
| configPath | string | `"/etc/flyteagent/config/*.yaml"` | Default regex string for searching configuration files |
| enabled | bool | `true` | |
| extraArgs | object | `{}` | Appends extra command line arguments to the main command |
| fullnameOverride | string | `""` | |
| image.pullPolicy | string | `"IfNotPresent"` | Docker image pull policy |
| image.repository | string | `"ghcr.io/flyteorg/flyteagent"` | Docker image for flyteagent deployment |
| image.tag | string | `"1.8.3"` | Docker image tag |
| nameOverride | string | `""` | |
| nodeSelector | object | `{}` | nodeSelector for flyteagent deployment |
| podAnnotations | object | `{}` | Annotations for flyteagent pods |
| ports.containerPort | int | `8000` | |
| ports.name | string | `"agent-grpc"` | |
| priorityClassName | string | `""` | Sets priorityClassName for datacatalog pod(s). |
| replicaCount | int | `1` | Replicas count for flyteagent deployment |
| resources | object | `{"limits":{"cpu":"500m","ephemeral-storage":"200Mi","memory":"200Mi"},"requests":{"cpu":"500m","ephemeral-storage":"200Mi","memory":"200Mi"}}` | Default resources requests and limits for flyteagent deployment |
| service | object | `{"annotations":{"projectcontour.io/upstream-protocol.h2c":"grpc"},"type":"ClusterIP"}` | Service settings for flyteagent |
| serviceAccount | object | `{"annotations":{},"create":true,"imagePullSecrets":[]}` | Configuration for service accounts for flyteagent |
| serviceAccount.annotations | object | `{}` | Annotations for ServiceAccount attached to flyteagent pods |
| serviceAccount.create | bool | `true` | Should a service account be created for flyteagent |
| serviceAccount.imagePullSecrets | list | `[]` | ImagePullSecrets to automatically assign to the service account |
| tolerations | list | `[]` | tolerations for flyteagent deployment |
42 changes: 42 additions & 0 deletions charts/flyte-agent/templates/_helpers.tpl
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
{{/* vim: set filetype=mustache: */}}

{{- define "flyte.name" -}}
{{- default .Chart.Name .Values.nameOverride | trunc 63 | trimSuffix "-" -}}
{{- end -}}

{{- define "flyte.chart" -}}
{{- printf "%s-%s" .Chart.Name .Chart.Version | replace "+" "_" | trunc 63 | trimSuffix "-" -}}
{{- end -}}

{{- define "flyte.namespace" -}}
{{- default .Release.Namespace .Values.forceNamespace | trunc 63 | trimSuffix "-" -}}
{{- end -}}


{{- define "flyteagent.name" -}}
{{- default .Chart.Name .Values.nameOverride | trunc 63 | trimSuffix "-" -}}
{{- end -}}

{{- define "flyteagent.selectorLabels" -}}
app.kubernetes.io/name: {{ template "flyteagent.name" . }}
app.kubernetes.io/instance: {{ .Release.Name }}
{{- end -}}

{{- define "flyteagent.labels" -}}
{{ include "flyteagent.selectorLabels" . }}
helm.sh/chart: {{ include "flyte.chart" . }}
app.kubernetes.io/managed-by: {{ .Release.Service }}
{{- end -}}

# Optional blocks for secret mount

{{- define "agentSecret.volume" -}}
- name: {{ include "flyte.name" . }}
secret:
secretName: {{ include "flyte.name" . }}
{{- end }}

{{- define "agentSecret.volumeMount" -}}
- mountPath: /etc/agent-secret
name: {{ include "flyte.name" . }}
{{- end }}
Original file line number Diff line number Diff line change
@@ -1,53 +1,55 @@
{{- if .Values.flyteagent.enabled }}
{{- if .Values.enabled }}
apiVersion: apps/v1
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this still necessary? If the chart is being installed, the intent is clear.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would using a condition in Chart.yaml solve this problem?

Copy link
Member Author

@pingsutw pingsutw Aug 9, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just realized we can use condition in helm, will update it, thanks!

kind: Deployment
metadata:
name: {{ template "flyteagent.name" . }}
namespace: {{ template "flyte.namespace" . }}
labels: {{ include "flyteagent.labels" . | nindent 4 }}
spec:
replicas: {{ .Values.flyteagent.replicaCount }}
replicas: {{ .Values.replicaCount }}
selector:
matchLabels: {{ include "flyteagent.selectorLabels" . | nindent 6 }}
template:
metadata:
annotations:
{{- with .Values.flyteagent.podAnnotations }}
{{- with .Values.podAnnotations }}
{{- toYaml . | nindent 8 }}
{{- end }}
labels: {{ include "flyteagent.labels" . | nindent 8 }}
spec:
{{- if .Values.flyteagent.priorityClassName }}
priorityClassName: {{ .Values.flyteagent.priorityClassName }}
{{- if .Values.priorityClassName }}
priorityClassName: {{ .Values.priorityClassName }}
{{- end }}
containers:
- command:
- pyflyte
- serve
image: "{{ .Values.flyteagent.image.repository }}:{{ .Values.flyteagent.image.tag }}"
imagePullPolicy: "{{ .Values.flyteagent.image.pullPolicy }}"
image: "{{ .Values.image.repository }}:{{ .Values.image.tag }}"
imagePullPolicy: "{{ .Values.image.pullPolicy }}"
name: flyteagent
volumeMounts: {{- include "agentSecret.volumeMount" . | nindent 8 }}
ports:
- containerPort: {{ .Values.flyteagent.ports.containerPort }}
name: {{ .Values.flyteagent.ports.name }}
resources: {{- toYaml .Values.flyteagent.resources | nindent 10 }}
{{- with .Values.flyteagent.additionalVolumeMounts -}}
- containerPort: {{ .Values.ports.containerPort }}
name: {{ .Values.ports.name }}
resources: {{- toYaml .Values.resources | nindent 10 }}
{{- with .Values.additionalVolumeMounts -}}
{{ tpl (toYaml .) $ | nindent 8 }}
{{- end }}
{{- with .Values.flyteagent.additionalContainers -}}
{{- with .Values.additionalContainers -}}
{{- tpl (toYaml .) $ | nindent 6}}
{{- end }}
serviceAccountName: {{ template "flyteagent.name" . }}
{{- with .Values.flyteagent.additionalVolumes -}}
volumes: {{- include "agentSecret.volume" . | nindent 6 }}
{{- with .Values.additionalVolumes -}}
{{ tpl (toYaml .) $ | nindent 6 }}
{{- end }}
{{- with .Values.flyteagent.nodeSelector }}
{{- with .Values.nodeSelector }}
nodeSelector: {{ tpl (toYaml .) $ | nindent 8 }}
{{- end }}
{{- with .Values.flyteagent.affinity }}
{{- with .Values.affinity }}
affinity: {{ tpl (toYaml .) $ | nindent 8 }}
{{- end }}
{{- with .Values.flyteagent.tolerations }}
{{- with .Values.tolerations }}
tolerations: {{ tpl (toYaml .) $ | nindent 8 }}
{{- end }}
{{- end }}
11 changes: 11 additions & 0 deletions charts/flyte-agent/templates/agent/secret.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
{{- if .Values.enabled }}
apiVersion: v1
kind: Secret
metadata:
name: {{ template "flyteagent.name" . }}
namespace: {{ template "flyte.namespace" . }}
type: Opaque
{{- with .Values.agentSecret.secretData -}}
{{ tpl (toYaml .) $ | nindent 0 }}
{{- end }}
{{- end }}
Original file line number Diff line number Diff line change
@@ -1,21 +1,21 @@
{{- if .Values.flyteagent.enabled }}
{{- if .Values.enabled }}
apiVersion: v1
kind: Service
metadata:
name: {{ template "flyteagent.name" . }}
namespace: {{ template "flyte.namespace" . }}
labels: {{ include "flyteagent.labels" . | nindent 4 }}
{{- with .Values.flyteagent.service.annotations }}
{{- with .Values.service.annotations }}
annotations: {{ tpl (toYaml .) $ | nindent 4 }}
{{- end }}
spec:
{{- with .Values.flyteagent.service.type}}
{{- with .Values.service.type}}
type: {{ . }}
{{- end }}
ports:
- name: {{ .Values.flyteagent.ports.name }}
port: {{ .Values.flyteagent.ports.containerPort }}
- name: {{ .Values.ports.name }}
port: {{ .Values.ports.containerPort }}
protocol: TCP
targetPort: {{ .Values.flyteagent.ports.name }}
targetPort: {{ .Values.ports.name }}
selector: {{ include "flyteagent.selectorLabels" . | nindent 4 }}
{{- end }}
{{- end }}
Original file line number Diff line number Diff line change
@@ -1,16 +1,16 @@
{{- if .Values.flyteagent.enabled }}
{{- if .Values.enabled }}
---
{{- if .Values.flyteagent.serviceAccount.create }}
{{- if .Values.serviceAccount.create }}
apiVersion: v1
kind: ServiceAccount
metadata:
name: {{ template "flyteagent.name" . }}
namespace: {{ template "flyte.namespace" . }}
labels: {{ include "flyteagent.labels" . | nindent 4 }}
{{- with .Values.flyteagent.serviceAccount.annotations }}
{{- with .Values.serviceAccount.annotations }}
annotations: {{ tpl (toYaml .) $ | nindent 4 }}
{{- end}}
{{- with .Values.flyteagent.serviceAccount.imagePullSecrets }}
{{- with .Values.serviceAccount.imagePullSecrets }}
imagePullSecrets: {{ tpl (toYaml .) $ | nindent 2 }}
{{- end }}
{{- end }}
Expand Down
77 changes: 77 additions & 0 deletions charts/flyte-agent/values.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,77 @@
# ---------------------------------------------------------------------
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will there be value in consolidating the format of our values.yaml files so that users can start getting familiar. Right now, all charts are very different from each other.

# FLYTE_AGENT SETTINGS
# ---------------------------------------------------------------------

# nameOverride String to override flyte-agent.name template
nameOverride: ""
# fullnameOverride String to override flyte-agent.fullname template
fullnameOverride: ""
# commonLabels Add labels to all the deployed resources
commonLabels: {}
# commonAnnotations Add annotations to all the deployed resources
commonAnnotations: {}

agentSecret:
# -- Specify name of K8s Secret. Leave it empty if you don't need this Secret
name: ""
# -- Specify your Secret (with sensitive data) or pseudo-manifest (without sensitive data). See https://github.com/godaddy/kubernetes-external-secrets
secretData:
data:
username: User

enabled: true
# -- Replicas count for flyteagent deployment
replicaCount: 1
image:
# -- Docker image for flyteagent deployment
repository: ghcr.io/flyteorg/flyteagent
# -- Docker image tag
tag: 1.8.3
# -- Docker image pull policy
pullPolicy: IfNotPresent
ports:
containerPort: 8000
name: agent-grpc
# -- Default resources requests and limits for flyteagent deployment
resources:
limits:
cpu: 500m
ephemeral-storage: 200Mi
memory: 200Mi
requests:
cpu: 500m
ephemeral-storage: 200Mi
memory: 200Mi
# -- Default regex string for searching configuration files
configPath: /etc/flyteagent/config/*.yaml
# -- Service settings for flyteagent
service:
annotations:
projectcontour.io/upstream-protocol.h2c: grpc
type: ClusterIP
# -- Configuration for service accounts for flyteagent
serviceAccount:
# -- Should a service account be created for flyteagent
create: true
# -- Annotations for ServiceAccount attached to flyteagent pods
annotations: {}
# -- ImagePullSecrets to automatically assign to the service account
imagePullSecrets: []
# -- Annotations for flyteagent pods
podAnnotations: {}
# -- nodeSelector for flyteagent deployment
nodeSelector: {}
# -- tolerations for flyteagent deployment
tolerations: []
# -- affinity for flyteagent deployment
affinity: {}
# -- Appends additional volumes to the deployment spec. May include template values.
additionalVolumes: []
# -- Appends additional volume mounts to the main container's spec. May include template values.
additionalVolumeMounts: []
# -- Appends additional containers to the deployment spec. May include template values.
additionalContainers: []
# -- Appends extra command line arguments to the main command
extraArgs: {}
# -- Sets priorityClassName for datacatalog pod(s).
priorityClassName: ""
5 changes: 5 additions & 0 deletions charts/flyte-binary/Chart.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -14,3 +14,8 @@ version: v0.1.10 # VERSION
# follow Semantic Versioning. They should reflect the version the application is using.
# It is recommended to use it with quotes.
appVersion: "1.16.0"
dependencies:
- name: flyteagent
alias: flyteagent
version: v0.1.10 # VERSION
repository: file://../flyte-agent # REPOSITORY
Loading
Loading