From 7ba306cfe6160d9ae63322747deeb2f8308a9bae Mon Sep 17 00:00:00 2001 From: Chris Doherty Date: Tue, 21 Feb 2023 10:09:32 -0600 Subject: [PATCH 01/14] Add tinkerbell CRD refactor design Signed-off-by: Chris Doherty --- design/14_tinkerbell_crd_refactor/proposal.md | 462 ++++++++++++++++++ .../workflow_state.puml | 13 + .../workflow_state_machine.png | Bin 0 -> 14834 bytes 3 files changed, 475 insertions(+) create mode 100644 design/14_tinkerbell_crd_refactor/proposal.md create mode 100644 design/14_tinkerbell_crd_refactor/workflow_state.puml create mode 100644 design/14_tinkerbell_crd_refactor/workflow_state_machine.png diff --git a/design/14_tinkerbell_crd_refactor/proposal.md b/design/14_tinkerbell_crd_refactor/proposal.md new file mode 100644 index 0000000..0f0ddbb --- /dev/null +++ b/design/14_tinkerbell_crd_refactor/proposal.md @@ -0,0 +1,462 @@ +# Tink CRD Refactor + +## Table of contents + +- [Tink CRD Refactor](#tink-crd-refactor) + - [Table of contents](#table-of-contents) + - [Overview](#overview) + - [Context](#context) + - [Goals/Non-goals](#goalsnon-goals) + - [Proposal](#proposal) + - [Custom Resource Definitions](#custom-resource-definitions) + - [`Hardware`](#hardware) + - [`OSIE`](#osie) + - [`Template`](#template) + - [`Workflow`](#workflow) + - [Workflow state transition](#workflow-state-transition) + - [Webhooks](#webhooks) + - [Data and functions available during template rendering](#data-and-functions-available-during-template-rendering) + - [Hegel Changes](#hegel-changes) + - [Migrating from v1alpha1 to v1alpha2](#migrating-from-v1alpha1-to-v1alpha2) + - [Rationale](#rationale) + - [Comparison with existing resources](#comparison-with-existing-resources) + - [Implementation Plan](#implementation-plan) + - [Future Work](#future-work) + +## Overview + +Tinkerbell's backend is rooted in 3 Custom Resource Definitions (CRDs): Hardware, Workflow and Template. The CRDs were developed as part of the KRM proposal that introduced a Kubernetes backend option to the Tinkerbell stack and mirrored the Postgres database schema (now deprecated and removed). As the CRDs were a reflection of the Postgres schema, they inherited the schemas flaws. This proposal attempts to remediate the flaws by refactoring the CRDs. + +## Context + +When users interact with Tinkerbell the primary interface is `kubectl` and Tinkerbell's CRDs: Hardware, Workflow and Template. The CRDs are hard to understand because they contain duplicate fields, obsolete fields, unclear semantics and, consequently, unclear system behavior expectations. + +Some specific issues with the CRDs are summarized as: + +**Hardware** + +1. Network information is specified in both `.Spec.Interfaces` and `.Spec.Metadata.Instance.Ips`. Only `.Spec.Interfaces` is used. +1. Disk information is specified in both `.Spec.Disks` and `.Spec.Metadata.Instance.Storage`. `.Spec.Metadata.Instance.Storage` is unused. +1. Userdata can be specified in both `.Spec.UserData` and `.Spec.Instance.Userdata`. `.Spec.Instance.Userdata` is unused. +1. `.Spec.TinkVersion` has no functional use. +1. `.Spec.Resources` was intended for use in CAPT as part of its Hardware selection algorithm but is yet to be implemented. +1. `.Spec.Interfaces[].Netboot.{AllowPXE,AllowWorkflow}` and `.Spec.Metadata.Instance.AlwaysPxe` are seemingly related but reside on different objects and how they impact eachother is unclear. +1. `.Spec.Metadata.Custom` defines specific fields that are related to other parts of Hardware. +1. `.Status.State`, `.Spec.Metadata.Instance.State` and `.Spec.Metadata.State` have unclear semantics. The `.Spec` state fields impact the machine provisioning process but nothing in the core Tinkerbell stack sets their values. `.Status.State` is unused. + +**Template** + +1. Template defines a single field, `.Spec.Data`. The format of `.Spec.Data` is entirely ambiguous requiring the user to understand implementation detail. In summary, `.Spec.Data` is composed of a list of tasks that can run on different machines; a task is composed of a list of actions that perform a function such as streaming a raw image. The multi-machine capability has no known use-cases. +1. The Template objects `.Status` field is unused. + +**Workflow** + +1. The `.Spec.HardwareMap` historically defines a template value used to render a tasks `WorkerAddr`. The `WorkerAddr` should be the MAC of the machine that should run the task. This creates a hard to understand relationship between Workflow and Template that users must understand to successfully execute Workflows. +1. The `.Spec.GlobalTimeout` is unused and its origin is unclear (status fields are typically populated by Kubernetes controllers to build understanding of the current object state). +1. Actions leverage the `WorkflowState` type that is intended to describe the overall state of the Workflow. + +Users resort to Q&A in the Tinkerbell Slack to determine what fields are required and how tweaking them impacts the system. The CRDs should be simple enough and sufficiently documented to aid users in understanding how they can manipulate the system. + +## Goals/Non-goals + +**Goals** + +- To de-duplicate Tink custom resource definition fields and data structures. +- To provide clear behavioral expectations when manipulating custom resources. +- To remove obsolete fields and data structures. + +**Non-goals** + +- To change the existing relationship between Tink and Rufio. +- To introduce additional object status data typically found on the `.Status` field. +- To support new technologies such as IPv6. + +## Proposal + +### Custom Resource Definitions + +The new set of CRDs will be defined as part of a `v1alpha2` API. + +#### `Hardware` + +```go +// Hardware is a logical representation of a machine that can execute Workflows. +type Hardware struct { + HardwareSpec +} + +type HardwareSpec struct { + // NetworkInterfaces defines the desired DHCP and netboot configuration for a network interface. + // It is necessary to specify at least one NetworkInterface. + NetworkInterfaces NetworkInterfaces + + // IPXE provides iPXE script override fields. + // Optional. + IPXE IPXE + + // OSIE describes the Operating System Installation Environment to be netbooted. + OSIE OSIE + + // Instance describes instance specific data that is generally unused by Tinkerbell core. + Instance Instance + + // StorageDevices is a list of storage devices that will be available in the OSIE. + // Optional. + StorageDevices []StorageDevice + + // BMCRef references a Rufio Machine object. It exists in the current API and will not be changed + // with this proposal. + BMCRef LocalObjectReference +} + +// NetworkInterface is the desired configuration for a particular network interface. +type NetworkInterface struct { + // DHCP is the basic network information for serving DHCP requests. + DHCP DHCP + + // DisableDHCP disables DHCP for this interface. Implies DisableNetboot. + // Default false. + DisableDHCP bool + + // DisableNetboot disables netbooting for this interface. The interface will still receive + // network information speified on by DHCP. + // Default false. + DisableNetboot bool +} + +// DHCP describes basic network configuration to be served in DHCP offers. +type DHCP struct { + IP string + Netmask string + + // Optional. + Gateway string + + // Optional. + Hostname string + + // Optional. + VLANID int + + // Optional. + Nameservers []string + + // Optional. + Timeservers []string + + // Defaults to max allowed DHCP lease time. + LeaseTime int32 +} + +// OSIE describes an OSIE to be used with a Hardware. The environment data +// is dependent on the OSIE being used and should be updated with the OSIE reference object. +type Netboot struct { + // OSIERef is a reference to an OSIE object. + OSIERef LocalObjectReference + + // KernelParams passed to the kernel when launching the OSIE. Parameters are joined with a + // space. + // Optional. + KernelParams []KernelParam +} + +type IPXE struct { + // Inline is an inline iPXE script that will be served as specified on this property. + Inline *string + + // URL is a URL to an hosted iPXE script. + URL *string +} + +// Instance describes instance specific data. Instance specific data is typically dependent on the +// permanent OS that a piece of hardware runs. This data is often served by an instance metadata +// service such as Tinkerbell's Hegel. The core Tinkerbell stack does not leverage this data. +type Instance struct { + // Userdata is data with a structure understood by the producer and consumer of the data. + Userdata string + + // Vendordata is data with a structure understood by the producer and consumer of the data. + Vendordata string +} + +// NetworkInterfaces maps a MAC address to a NetworkInterface. +type NetworkInterfaces map[MAC]NetworkInterface + +// MAC is a Media Access Control address. +type MAC string + +// StorageDevice describes a storage device path that will be present in the OSIE. +type StorageDevice string + +// KernelParam defines an atomic kernel parameter that will be passed to the OSIE. +type KernelParam string +``` + +#### `OSIE` + +`OSIE` is a new CRD. It exists to ensure OSIE URLs can be re-used and easily updated across `Hardware` instances. + +```go +// OSIE describes and Operating System Initialization Environment. It is used by Tinkerbell +// to provision machines and should launch the Tink Worker component. +type OSIE struct { + Spec OSIESpec +} + +type OSIESpec struct { + // KernelURL is a URL to a kernel image. + KernelURL string + + // InitrdURL is a URL to an initrd image. + InitrdURL string +} +``` + +#### `Template` + +```go +// Template defines a set of actions to be run on a target machine. The template is rendered +// prior to execution where it is exposed to Hardware and user defined data. All fields within +// TemplateSpec may contain template values. See https://pkg.go.dev/text/template for more details. +type Template struct { + Spec TemplateSpec +} + +type TemplateSpec struct { + // Actions defines the set of actions to be run on a target machine. Actions are run sequentially + // in the order they are specified. At least 1 action must be specified. Names of actions + // must be unique within a Template. + Actions []Action + + // Volumes to be mounted on all actions. If an action specifies the same volume it will take + // precedence. + // Optional. + Volumes []Volume + + // Env defines environment variables to be available in all actions. If an action specifies + // the same environment variable it will take precedence. + // Optional. + Env map[string]string +} + +// Action defines an individual action to be run on a target machine. +type Action struct { + // Name is a unique name for the action. + Name ActionName + + // Image is an OCI image. + Image string + + // Command defines the command to use when launching the image. + // Optional. + Command string + + // Args are a set of arguments to be passed to the container on launch. + Args []string + + // Env defines environment variables used when launching the container. + // Optional. + Env map[string]string + + // Volumes defines the volumes to mount into the container. + // Optional. + Volumes []Volume + + // NetworkNamespace defines the network namespace to run the container in. This enables access + // to the host network namespace. + // See https://man7.org/linux/man-pages/man7/namespaces.7.html. + NetworkNamespace string +} + +// Volume is a specification for mounting a volume in an action. Volumes take the form +// {SRC-VOLUME-NAME | SRC-HOST-DIR}:TGT-CONTAINER-DIR:OPTIONS. When specifying a VOLUME-NAME that +// does not exist it will be created for you. +// +// Examples +// +// Read-only bind mount bound to /data +// /etc/data:/data:ro +// +// Writable volume name bound to /data +// shared_volume:/data +// +// See https://docs.docker.com/storage/volumes/ for additional details +type Volume string + +// ActionName is unique name within the context of a workflow. +type ActionName string +``` + +#### `Workflow` + +```go +// Workflow describes a set of actions to be run on a specific Hardware. Workflows execute +// once and should be considered ephemeral. +type Workflow struct { + Spec WorkflowSpec + Status WorkflowStatus +} + +type WorkflowSpec struct { + // HardwareRef is a reference to a Hardware resource this workflow will execute on. + // If no namespace is specified the Workflow's namespace is assumed. + HardwareRef LocalObjectReference + + // TemplateRef is a reference to a Template resource used to render workflow actions. + // If no namespace is specified the Workflow's namespace is assumed. + TemplateRef LocalObjectReference + + // TemplateData is user defined data that is injected during template rendering. The data + // structure should be marshalable. + // Optional. + TemplateData map[string]any + + // Timeout defines the time the workflow has to complete. The timer begins when the first action + // is requested. When set to 0, no timeout is applied. + // Default 0. + Timeout int32 +} + +type WorkflowStatus struct { + // Actions is the list of rendered actions and their status. + Actions RenderedActions + + // StartedAt is the time the first action was requested. Nil indicates the workflow has not + // started. + StartedAt *metav1.Time + + // State describes the current state of the workflow. This fields represents a summation of + // action states. Specifically, if all actions succeeded, the workflow will succeed. If one + // action fails, the workflow fails irrespective of previous action status'. + State State + + // Reason describes the reason for failure. It is only relevant when Result is ResultFailed. + // It is propogated from the failed action. + Reason Reason +} + +// RenderedActions is a map of action name to RenderedAction. +type RenderedActions map[ActionName]ActionStatus + +// ActionStatus describes status information about an action. +type ActionStatus struct { + // Rendered is the rendered action. + Rendered Action + + // StartedAt is the time the action was requested. Nil indicates the action has not started. + StartedAt *metav1.Time + + // State describes the current state of the action. + State State + + // Reason describes the reason for failure. It is only relevant when Result is ResultFailed. + Reason Reason + + // Message is a freeform user friendly message describing the specific issue that caused the + // failure. It is only relevant when Result is ResultFailed. + Message string +} + +// State describes the point in time state of a workflow or action. +type State string + +const ( + StatePending State = "Pending" + StateRunning State = "Running" + StateSucceeded State = "Succeeded" + StateFailed State = "Failed" +) + +// Reason is a one-word TitleCase string indicating why a failure occurred. It is not restricted +// to the values defined with this API. +type Reason string + +const ( + ReasonUnknown Reason = "Unknown" + ReasonTimeout Reason = "Timeout" +) +``` + +### Workflow state transition + +![Workflow state transitions](https://raw.githubusercontent.com/tinkerbell/roadmap/b7b2362997c01ffa52758e10259b8f55e81f7447/design/14_tinkerbell_crd_refactor/workflow_state_machine.png) + +Actions follow a similar state transition model. + +### Webhooks + +We will introduce a set of basic webhooks for validating each CRD. The webhooks will only validate the `v1alpha2` API version. They will be implemented as part of the `tink-controller` component. + +### Data and functions available during template rendering + +Templates will have access to a subset of `Hardware` data when they are rendered. Injecting `Hardware` data was outlined in a [previous proposal](https://github.com/tinkerbell/proposals/tree/e24b19a628c6b1ecaafd566667155ca5d6fd6f70/proposals/0028). The existing implementation only injects disk that will, based on this proposal, be sources from the `.Hardware.StorageDevices` list. + +The previous proposal did not outline a set of custom functions injected into templates. The custom functions are detailed in [`template_funcs.go`](https://github.com/tinkerbell/tink/blob/main/internal/workflow/template_funcs.go) and include: + +* `contains`: determine if a string contains a substring. +* `hasPrefix`: determine if a string has a prefix. +* `hasSuffix`: determine if a string has a suffix. +* `formatPartition`: formats a string representing a device path with a specific partition. For example, `{{ formatPartition "/dev/sda" 1 }}` will result in `/dev/sda1`. + +These functions will continue to be available during template rendering. + +### Hegel Changes + +Hegel will undergo a reduction in the endpoints it serves because the data is no longer available in the `Hardware` resource. Minimally, Hegel will serve the following endpoints: + +* `/2009-04-04/meta-data/instance-id` +* `/2009-04-04/user-data` + +## Migrating from v1alpha1 to v1alpha2 + +Given the relative immaturity of the Tinkerbell API we will not provide any migration tooling from `v1alpha1` to `v1alpha2`. Users will be required to manually convert `Hardware` and `Template` resources. `Workflow` resources are considered ephemeral and can generally be deleted. + +## Rationale + +### Comparison with existing resources + +**Hardware** + +The features available to toggle DHCP behavior have been reduced to 2 options specified on the `NetworkInterface` data structure: `DisableNetboot` and `DisableDHCP`. This is in contrast to the existing `Hardware` that defines 4 different properties across various data structures including the `State` fields that require specific, undocumented string values to alter behavior. + +We rationalized that a machine can only boot a single OSIE at a time because operators do not regularly change the OSIE they want to use, and that there are weak use-cases for multi-OSIE support. Consequently, the ability to support different OSIEs per network interface has been removed in favor of supporting a single OSIE definition. + +Numerous fields served by Hegel have been removed with only the fields necessary for honoring cloud-init remaining under the `Instance` field. This significantly reduces the data available for Hegel to serve and _will_ break some Tinkerbell maintained actions that leverage Hegel data to configure disks. A separate proposal will address user defined metadata and how Hegel will serve that data. + +**Template** + +Templates are explicitly defined as part of the CRD in contrast to the free-form string that required a specific, largely undocumented, format in the existing definition. This will signficantly improve the user experience as it establishes clear expectations and feature support. + +[Tasks](https://github.com/tinkerbell/tink/blob/main/internal/workflow/types.go#L13) used for multi-worker workflows have been removed. Multi-worker workflows have been unsupported since the transition to the Kubernetes backend. This, subsequently, removes the need to specify device IDs (device IDs are specified on the `Workflow`) when rendering templates simplifying the `Workflow` and `Template` relationship. + +The `PID` field has been removed in favor of a `Network` field. This facilitates known use-cases that require the container to run in the host network namespace. + +`On*` fields have been removed. The `On*` fields provided rudimentary event based mechanisms for executing arbitrary logic. The semantics were unclear and use-cases unknown. + +All fields on the `TemplateSpec` will support templatable values in accordance with the [`test/template` Go standard library package](https://pkg.go.dev/text/template). + +**Workflow** + +Tasks no longer feature on `WorkflowStatus` as they pertain to multi-worker workflows that are no longer supported. + +Reasons for failures have been introduced as a core concept, via the `Reason` field, around status reporting of actions. This enables programmatic failure reason identification and comparison. A future proposal will address how users can provide customized machine comparable reasons for failed actions. + +The `Workflow` provides a `TemplateData` field that can be used to inject arbitrary data into templates. This facilitates users wanting to model variable data on `Template`s that has per-`Workflow` values. + +## Implementation Plan + +1. All services that have unreleased changes will be released under a new version. This provides a baseline of functional services that can be used in the Tinkerbell Helm charts. + +2. Develop the code to leverage the `v1alpha2` API. + +3. Hard cut over to the new API version. This means versions beyond those specified during (1) will no longer support the `v1alpha1` API. + +## Future Work + +Introducing mechanisms to propagate reasons and user readable messages for action failure. As these mechanisms do not exist currently and will not be realized through this proposal, the `.Workflow.Status.Reason` and `.Workflow.Status.Message` will have minimal benefit to the end user. The separate proposal will address the contract between actions and Tink Worker that can be used to propogate a reason and message to workflows. + +Maintainers have informally discussed inverting the relationship between `Hardware` and Rufio CRDs. This is necessary for defining a precedent on how to extend Tinkerbell functionality without expanding the scope of core Tinkerbell CRDs. + +Introduction of user defined metadata to be served by Hegel that could facilitate user defined actions. Similarly, injection of additional `Hardware` data when templates are rendered will be addressed on a separate ad-hoc basis. + +Separation of `Hardware.Instance` data into a separate CRD possibly owned by Hegel. Given the instance data is unused by the core Tinkerbell stack it diff --git a/design/14_tinkerbell_crd_refactor/workflow_state.puml b/design/14_tinkerbell_crd_refactor/workflow_state.puml new file mode 100644 index 0000000..3b8c2f5 --- /dev/null +++ b/design/14_tinkerbell_crd_refactor/workflow_state.puml @@ -0,0 +1,13 @@ +@startuml Workflow State Machine + +[*] --> Pending : Workflow reconciled +Pending --> Running : First action requested +Running --> Succeeded +Running --> Failed +Succeeded --> [*] +Failed --> [*] + +Succeeded : All actions completed\nsuccessfully +Failed : An action failed + +@enduml \ No newline at end of file diff --git a/design/14_tinkerbell_crd_refactor/workflow_state_machine.png b/design/14_tinkerbell_crd_refactor/workflow_state_machine.png new file mode 100644 index 0000000000000000000000000000000000000000..2470709d63cc8dc17030b66532a2c0b1aef03c4e GIT binary patch literal 14834 zcmd6ObyQUSyYI}%AP#~c3`j{SoeCmI3`k2$gLDZ}N{5UhB@zZGqJT6AN`tf#Vjx{2 zEg{_<_Zi>cIqRHz&RX}Zd;htAyz7-cdw=&gpXU?Lj?mIjBE!&Q5C{aBin6>80)f*(#}JYUv->f}*UyzNJySH8(V_I`Lc@b&cepv>Y91wwX1e(6NUYHv23S3Jk& zYQRkK`BZ_tk(-gMpQY%9^K`U^^67EMcE{6iwbS)Aqio}O|Ew^FH4|5!N4>t%Oc8ms zK<7-HJZasR6`{<2W42BU!_KM^P5dIa991(}C4v;r^pKlxW0&L~^qKr3#CEkVQbtw2 zMMvvq-vG)hTOChalb(F8rb=Tf^6^_16^q#P16jYJuoWA$?BbIjUAK{n;;j@Im%OQD zffpWMUM)IYj-3?Eq{#ijGbbnQoG27`E=_^Kr6FOz8KJip60&$sBMP63d z=jXy}e?x4~@aGyQ-)|xf3<)IMY?T*vAFcXzUN`Fqqw-UZ4(l}F6{Wzwci?FjrlyR+ zqs(H|K!)WA3uIbtrvLw=}Z)JElSBoq<&94N|6or>u$s& zkKMj=$H)}g`n*lMzBI!8t+*X-VB}DH(Vlk6V(0j&vjxB40@0JFPyM@-8XM(ln6S8K z^)H02XH)QF4PnJI*$AsO(+{^Zlx}>iqaZOcu|xH6 z42vqejLb-j3~92Ie|(byngY!hdQzC;D3VriDi-EJ`)(TkSeQ2Q!9K(B=-y;bIvH>5R#Dnt=k#hQotBGUi&o!7^v3_LPtB>kd`X6r_D9(;GLA_NQec zpT$V;lJrf^4-_}eCrSBBc+B7#w%K`Dq@`>1-sdsV!$i<T9t{J%cqhH+ug zDLEtA+G#U04yDFXC)~Xy{MKj0MgqDDEU$`8FiZNR=;XNdu%X;Vt*z?u$W;B9$g|(7 zSDG2Z$eDJhWsbP55IZcl6(5~(KROKLyRM@%zOXPV&(GiLc4$QE>*>ilYfA5&`?)aPmVC6{rOB*o zJ(}`T>-i%NE%^V~U$v1q0%E(1mAv8cT)56#~1>OaG-XKAp8 zdCv9P+SxsvrWY3%XA|b)Qoh6AaD1cln&83eH^CQ*tQKiHKa3M*uH)xjaK~SSM@Cv- z#T&jxx!HQpv|Fx>H&JnFG|t0`dF0O`{rLXC@8342dFDX}rG{tR?+?Vt92%ByL>lRC zuPnf5M8(oIk-N7Qm#-rIH|GJksnLkvrA zD&Bu@se{pqM7d{am5>7U*E~;=`^`3KFby&^GyvxH_#TFaOkrW*c}vV ze&SIle>2k1aJZzusP&xFPg9H%isUGCq_D z#_ZOIoOcd(HyU0WrEe_^ZoT#%eoXV!j`D0p?_jyBxuIdWkuF(5{Gt`{ZE5dCMjh0# zV@-KsVM?E)nLJI9m3wY~Iy2r}4YL{wC1WUa>Ras6lsU++965WN!sHJ`HLL!X;GY%l z6B98q0-3fKJ4_Al7<-95dGdsX`kaKgIKN>59>b!ajl}Fd7KB|orntO|C=H(EDta#C`hP~AF>fVc*q+6y7YgW zMZy}0rd9^Lr)vvp;9 z`U3@dG)2h7?Ce7a+NBaojPkW>4f1{OZWvvL>?UnwWJKOmkJ^^s_nz%OLFGVF;08&T z_7UAh;;sE3udlGLt_);cd#;_PSZD}82&QLdN?f^eWj`81rK&0>G6ci2DW|6wr0CqrRbY9kgDgSAri|XKYskk zhk?Y-e*XM>n zuv4mhR@gKdvADBwds|DNo0|oUiq3I!|JmPJb7UCXTdEr=b?O^$iaf=5x&Em#I#%NL z)c1rCQrf$F3+0Z9=lS{hg@j`3E^8svdtt67e!MRWd}m%4J&QOBhP#a^=;(acXS7Ro zxC8~m(1x@rsBrD{^mIlE59NG5Em{%`X8cPAN}0^G%8Q?kO)g))v$Ior=QmZ76%YOl z8wbbKRBIy5i4H<b}>{TR%k*OjV3YHvT?^VXox zAmEbg@cxkZ2)2t|@zL`7dY(6WeyHmCYswr%Dy_V7lnj!b@p6Gv-#ej@Q26IMmI#IYrxqW%;W713R^D}k3ma)dlm8@N|y zXt%rmM_O7M;+i4p2y737ud^`9?ac_zq?!K03CM{rFFEsYZ&u~xB>S3;Jk zcp0zx7u16!dvyR}PMkP#_UzehRUJg{r%=c5V;gdTFV#5p2?z=bwlWe9*b7zU zYY9)krJ5~(B9ymb(_o!%J*deh$4dw9vanL9=)y`!9ZgbC7RRzSx^@qAck# zL&4Vd@#7m!QGS!MfT;v~kKmoGZp)*xYC+2%j~9t+ zA$1YLzE^SM_4kg2EB@+a58}LZsn-p$G*ZXh6?C*WGM9&+_#9*H2be||fyMFVA-caX zT)K7HZ{1;Nas%yl^CtJ~0t9+$b1XG=N|`w?+M@Dm82Rw<@c7&9Z69e;*rm4Ah)j*w zgs8T&Hk~ZtIXlyl*12%Q91;B7%r6I4^R0e9K!DpG9wN(^wam%8GSwUMR5R$M1AGAS z3q>*g9kLwyB)v$>(9jJrCT~v}8JS;SD8tkfFDa-!T(@ipwZyg}$WNT0ELMtU{L^2^ zu47#8QUf89hTm8j({B0=N8b0|958iRSlDr*b%_09j@=FriV1^K%SJT(s`TXLiMk|caMFD==6bnqrbGg`6&8P@bkr93&u^n78|oYe-9idD=Vv* zn8C-DAb=FGCzk)7tJT$2*fKnZNJxTQTwJ3OgbwMn+%OuAyV;vvYGXWMq1V`<&;`+uGZ&{f5-Z z1wsEgze(ncFjPW90;I+1n-G{5_zE0*c+e#z(Tw7!>F6SI$=bNFIL9Hj{vun_-1aht zBS`tG>gtm@=_utPZZ0l7Norc!+S=OEP?i0paM0G`5bP3CT7kx-IO@MQ z<5w3nDk8!@0L()}OUn%0DC4=AV{e|(bAcPcxC`s=qe)+ixeix)I?>WT2kLdxHY9+U zjQ#=o*U(VRSt*{=@Bf`_urfL6>FdwhrG7YN@XjEz`{?_nlZt5D4! zL94`&5p7y1hS+<49X))Ad4XDqfBpL4$CagUIb1bs5MO*!(%t!f;cQjRCymz<`G$pZ zzl%Yxa2cw!L?!mN5?wK}{?hW!!D0>q^HbE%lACL$q7HYz@mPF+A*@4Ic4zuWIL@@I zG`E3$lSa_&(H?82=4*+Nw~}03EpD%keue>?vG(nbRoI1`APj`SsPLpqb>03dX(nK- zNQRs|RQdsJVrU#o6@e9Z0YM>AR)Z7uN59Dy%aRx&eE#}i{&k3UWJrFra@V+oVkymY zh^W@CumEW&IFXd%+1Xn&9ap(n#Kq~}P9)t*M>Vbz*om^zvh^ka*WRDy;o&5KnAmtq zBTLQQeR4-GHu}1qDF*ic_yT$?%PgwGt)s-O#Nn~Wo4uW#KQ5ENd7inrkByDVAvraZ z})P|^n(Wv z!q;zhW!~&Xm0S?cv~jHRr_Xl7Iu($16U!LzIr~?2V37pb+C*@Bx zDZR7LgTm*}4gO*9CVF}yG=g3T?LheZ*m3gx{k68r1m(f}yu8NORBxw&l1xeZERTYw zz*b0(p@?C1doO+aH_2_B%X2|<#mcla_En_=Nmf=Qk`FxwLjw{|!M8c=zr?D|1q{eq zHM7hl3l-i(fJ7tS!@f%1yD-Ey>AhPGp=2On9fry5kL_;Eu`zA^womfSRC&swlcP2n9}OgZ5X|v>V4_h@J;}cPwrEUc+K}yUW=v| z#Uv0YqLs~;?urdGDGemSJWTenimaR-#~}%!#e1jDO19sq&C5F}P*0SLc*#2+YGh{S zV#9$Tn`aK%FEk@(uF^I(&P})t62OW{n!eNibMw0$KUIz;oH4pUTxM1WEuO<@UE0{g z{U{=&wrtUXfq`$&1r3@D34azi{gde)5K z4_t%3qod=`pFc&ImhaymMMPLqYJ-gUEIyuXKI6HKDkeN8=Cg;2%85u2h? zk3Sb2ySZpcfn$Vpck9~Q+lP$+CZ=?4f?gDo=H}+=^2BAM@en*78U#`TlpuwkjelT~ zW@l$NA_$0r-HxSCpHfk`*Z6x6d0P;;R`UOZtvLQywzBcRu@z=4ZuSQtcA)6b`HY@C zd**St?^~+AaafFqK^!orYqba`LK#!@6X@Zejtw6!U*Sli2YiBFyVmHiUZ z#UpxA9sz+L9)%KCjU<|yZGLTDAiB-RKyss_U=Tedy6iSu59{=a8Wb7lo?NV{+iW*C z6*tS}%Ra-^zL_(Eh&wADKr|9J=KA{1$Y|k4Y6F|rpPau&FX8}H%jhjh^_^)ArrO*F zE8_actAB6(%Sl)`IEXR*AMRwmU;;|g>0I%F2VjoDJ`&*Z{aAl))EC#R>M%NvB( z1C&&y>)qCiiX5tNKQjxE-0f7@LR7}8@vui-{mx9K0RS+9nQ zIR9L4jG!9l1@&8`sY&Gih$N=o0|dnCnwlm9hZoC6aOA(gK`g@n%N;sW4c?pC+ShpMV;#c$rcArW7p zx}m73*p?(E9C=wyL!-GdjQK7X4G@|$`rRi-0qGkD+FBICK89Nc8yc)Z_e~Bu^aVmF zlE_JDaP~SKdzZ@;k=||0MPn{!E$d8kNy7# zaxg;BQk~|HF>-5){kPh115M=-V0Y~_110sc2Djtaa+TYC#sq)t^)27cyO_8+a zw+23SsiP^Jf%gFC%Jo4rUUmZ&9#7bt7jQ_5dqvm+;HQ5N;~fGyI6hW>^WVq;oX{Ct zknJ7#dG(NyH2k@FdB2Sav;tzA+|{dCx+5Uy5ZDPPKIb0D5)iz&=ilJ$?504#^u3*f zu*yU0=No6dp!o>74MK^hpGq2)1q46}^Ok0`4<{alNid_jOB zUewpux3_Dpxzc(ffq+W?299xFHqalaIn5u};3z+T|FqJ!V#0Qaef~U4Cx>slnk+xi zz@V?BL?nq6fGR^tgOrq1S63H*6$3D%0-TW%T^ppd?+@$hf zac<5@DrP(&AOO%x(;CQFcVgq>CdbAs_3A3$_BaX(zJ~!NF+syLC6$y%`rFT|?%zjw zdv8@&OD3_yWlZ(^wY z1$7kAK$hde^tAn*8!LqkK>Z~HbrHV0pBfsbzb9Bn`jg)=QaMt9yv73y$gUJ!(vdu{ ztp)owMqXC-p6wBdq4_Das;a8bCE#;cK#hWU=LH&tiA*f?3)Gau2*|;uDY>~{M@E7M zifq4Leg#JrLhEV?%#sQc?BOgATpN<85 zG_$z)<2ktw^7qKnZ#tQ~)%Tf^v(_*MBoZ0GOoz2tdJr0Fykls=A@@ko=5aY8cN#0J zs1y##prj=(yE*x=o(9?Fs=hwW&xT6}O7%*doSer)Qd-_=0YP}+L)4NrUZ)omAq0X9 zFEdRXhGpZ|$<58pDWR&() zb7hZod9Z{_T(6!yIqmK3T{Vlgkpx@>S!W^{B;2M{{kQHB#s1Yuko4k2^g+-#JZ0Kj zTU$GS;yOEsZ&HBCgjR8h3YgR$}lQ+ghWgeP$EvdOY>H9gIijmaar4PmHm~dY@yM?X|U#y~?(> z8NtE9+di`%5d~`hZd81`(pGXD6N4>qghRd88=2E9*_ajKu`PG@O$+Vv4saMX6G{;;yERZGbGQ;i5K&i}XzA8Gb#ac4Jn!Cs>r-WQ@tisbdNH`ud9U@&z#| zg&0EsQhGNs9b#Td`d*NcIRb6BD3|GK)9EP3>+h{}Tt;f0WscDSfhj81Z~Z&P00#4> z>4(ak2Z5Cjrx0K>qohx}-)265G%)Jk&!B*x^~4$%l1pyb`^Xum!Xs|joeEI}ok3rOtnv#aHHwCygE@1m z)2AWI#lE3#^E7Po6Dmga&o$mUvv^?1fT_4W`IY%_`MA8sw|IV#n?NZC{NX&KiQs|0tW6yS!d3u93*ID#yTd zucf`;3Tx%U5kkW+Il$G9e`Vi#k;RKI=jfFi`6X7p+ubh2)&?EjO+PwV3hIlR{`+h} znHgv4H~{)=XT0H6O^U!8nOkbe`hfx16+f(gwzjJ4mLcz(>(m2*&@!qof64om1CSM) zJEUM^laA+R#>SZADs3#J7z`NoY=l+m&qC$B>H9+Pm=1zmepOgn>`<~RhHFP47jYo2 zJD$?q1ydvHk5I0L~nxp46!sJz+b%%(6gB1q@s4iEi0EEI^_yvNGP?gGOEoTZf*TfAr`PLFTD};QZyw zmlt^UMgs`DuSIU2o{w()eAXW#mF@0n_vf2yYYngIB48^rNe5JTY{jd~-2SZ$N8@on zNQ|&$fyT&5RKYTnV=c&J#MwhT9*QiFs+8G>Nu&E!H7N#$naesTaN1K-Q%RD(I%O%^ zrM>0jz){y$S4)Tl6Mg1-^IyGU3~-!eNsRVv1Ix?IOrWYmB2Nx&7(tNdppc^n*icrI zpCgscHDI!2g%|e^4i=P< z!Ng=w4irW;gLbb!bwr_1W`P@eImLvspn`{nig$ia`Q0y|0dpn$-uSo`QCVfBm?|)V zd(v{JFLhJ>m5kah5QtQgEx}2V{PvHll5~zBGR62j5&5j0)9`|4GvW^>*TuRUR3}_3 zCpRZ&7D&-hU1I!aS3;wk_N8?h7x8uKXDKd2+O3(0O^L$XVdm27^`MtD$-LD_lDw&d zz6C`QLPAv;u+#r8G?% zj((V)cyPNL%#~>;LTmB5Mr!lSt5pzOYxY;a8LD-f zNBxK1A+B=%fm)Do0it7>l*DYtvzvUlJCZa5V*4}t_zX!xVvt;%W;=jPj5D;dyeTX{ z4-#E>-O<3Eoz=X0E^Cfv;+1${od8ONN~HMZ z%gV>5QhDxC^hj_82DdrH1?T*o9e6+YrAzvcQetnA2{`FGWzc@2OG6B-7}|_1TjP+a z!uJPL*0D5COU0h%63sC}@a@t%NqFh@=cEV9{^H2)-D!iyJNc7cI`8DdNh zam2YR4h~SIsNmcz#>NDG3DDyE4g&;jGnI`~5 zuwV1Ek+ZWOTeK5s8>$_gHN3efKJ8gt<|xKgQzo?(bl*d&t~?|i=a;H7WU6>su6 zjl@gp>1p)$U)IgL%<$mOrN%`|Ddn48yz#F$&WzN_NM>)u)qb=u`TDWjy4%ncx$F1( z#l77HpZjx2dyE17LtSqRoq?#A3mfM=b?d*z@xDR)K_(&k*iJVM);j6p@BN*|Vyh<1 zxy<(AJQ5|L%27sf;}hr-31% z+a{q{e5eFDCn!@;2lbJTnmRo(_@54?|9gi}PrW25m&zKN?0keHs|l0f=II$ySh6a2 zp6D~T8R;8WHLUtSQ=JT){1;g)+b~ORg1xqO2Y6f-Ml0`DK>t)tY&TWoLTGbRFaNsJ z7NMSk<^NAY0Nk1&gl3@Z3i&ApNLT#ZM(`5H(Ng@#DS9Cq%rd$Wth z%%Vx8(L|D^EJZ28R!J3h)aR-(+KVdF)lmWesIzEt3I;hXndj?2ux74Xun7`RBgaCN?Hb z4+d040zkZ8jmhKEos7!iTd#Xpu3&PMQR*Tb0tALP7qxqgj?NR_{X<;Q4F zV?Og*(={XfM)e!zUo6Zj$mWwDEWh(qMu>Vn5Q^m*qzc9q7^Cs<=*w{mzQL7rmn)eu zeoF`Ldz;Q&xGmE&ZxI8mV(lqU=sC6gA@*u@JR!$<5s?qc2QsnRlxHVq4^lPMn|w*AoM&GGkq@yYzuu--<1ES zPwVK!TR|s}MHHpkwzz0t(w?*L|H(57GIpnhG{Lr_;uR5jh7l6=UK&=2)7m;RP9GL` zA8!OmvHqIG8q0F!cWAA}(}fsI7HzMRlLP!;IiyojPyl+YT~d~G0U)D8XZndodGw3v z4=jB3m3Wl_a4l}#x&@Bv0(Ai#$f0VFuwU)8w~7vgPoDc49sj2HVoR9Icp5LF^&DSl zFLy8080ImVaRCFkYBDcY99)SII3Z0dg#Pg11DA>JZL(yPFPep1H0VQ5085Ve@6g1= zL|s?fy6+K$Zm}jhSp3s1M{eAR8IZHtBY<7Bb*BT@JCwY9K8%I}0z60z!CmwO)ekuI zlbTuO{0g6yaY%*Wf!wDuh9WfUv2VOAGBV~DPp^}G7+YEz0r2?xinrV)64us#Uc)s= zDd!cr>vi=Sl6doLea!u*9O|dlPyXRNQN<=8@UEm}T`Uqs*7W&vd|`z>-oO-9rB1@#DuG#rB%#yyklO!CR%u){cB{vuEAH z@zv`=>`-d$Q2T6zrN#S`C!JA~U%WWVkAqt4jy%Y80HoX$Yg0#QO^qmX8QNVaA8|C} z+70~$A-Ub#`OD_#SoLQMgxzATcqmkDP$aLYn5uGn@t61&U4)|=3>Q=T)m^xvvJ$qP z@Tk)pJk+nH`Py*`v&_=!XsY=;Nu<6>AHBxdqXyc?K(u>fEUF$f8k?|z>u0l!z@LyAF6IU~iCLpZNtE=<7qJM<<(%7ndw@T=Rxf%-PB7`WekZZf|<;`dH;Kyl? zrjMWkD`;$SFQH@vQR`=EEa;sjk>#mXM|sZvL>aOw6qyYb*+v#Q9o~oPr>e?I z8Z1`0dd?h~?vipoB(eyEaRp09n04)L&?*Q8gL>Dijy=)K(alB|aAhN$h@6qGuCC}K z5?7;cN6~YMuU>s8s64bmK|!D#AFAms;qstn!H$-Nr~aqx@`UP*Vmq?P)VtSu`ufD6 z3eY73bqf#XBCtNA<+h!Nw=89mkAisW?YOwP-Mzi1nfqqZx;fhqLNEnV(2X0WrbWfY zM$|E%Ku_1s7+>Pk#j8Jg`EnM7hbtvlgo$<2fUVBIDW<}v7@nr0@}+vq0ITkx2IgEL z^f%G<>+bgUnTy}qA9L~W=;Y`+s1Yuo7jo(agXL67RC-?C_pYw%BUSDsSPTCY%;>^` zl=q@3jY3qo0FDv;2%|-xdgtxV$h5TyTmY*D<^*>Sw1~>8s^Z8hq?Wc^S_D4_53Z0P z+Er96$G8*~kWyeRXw$Q@V18H21^D~X;g|w;=$#%9*1uZ$@6R!D!(a%kE*R7m67DW1 zA|@s#BI@5s6ZAr+qULE=jzRIzxA`)nzYlxQe_%LCFFTKtN5=+v(=0D97nDCnKiDJu z$a0jKl2Q+t^PM6M@D&S~dHg;zV1WYs>wO*$Gols&oGxM~2&5$+Kfn06IPN_mb~d)Z zkNPz$Kg^z|9!L@~vwHd%JVg=2?@hd>`OSF$0?H_Dh3LA~b7 zoZMW$h0?xf-L}deNH{ptP@NgBo!#SDk^sgo){PHD`FRwfvNQNn@!Pj=^Od5{@ZN-| z_t)bP1(~`4zYs0RW-P+U8--!uu{g-4oFP3|(%yEn);n1;u zEPQ<5%bIB;!o%yyi%EsxrnO8*-uLHJH8eJtmqB!#1T8|jhP9ct@G#R8l(Z@Y$j!Pm zXQcc@X8ZN4!l2L^I1Qmw6xRuN{dnw!!C>5urg=|;EpnUG&@w$j03VS>r z8=b_z@8G}{Z|LcXBP@g!Kg=db)&naFXERa-_CBY>Hv_b^w6!G9pqM;l=GEIP(j#e) z`k`V!H4e&q?(ZFa3_fl&3XMs?;esGsGCLh};oOJ1y1FSo<3s&HsRS+LZ<#G;s2|=@ zx^et?SY;`ihnKhT-PHW{%ESOV7TWzmD7qi%4-wK(Q_H(5oxN}k14VF9IyAv$0WFh( zUgZ9nX6&I5u$5a(7i8nDEs8cop*%5A>038G_hex8ro;!Z<6B~Se# zO8Vl(i~RigDJi&0ZFVd0i70DGCV(fWd1?*nXM9h>cVV??PlI^4MaPMcJ@@0wk9V?)DL8!QDyEVwu* zC^`|<{xvjXUXm!9o8OtjnA)%Te36k|ZCBQQVM9&6P#)2~EIoquw zFX*_@d?bLsqyTDKM$1xf{$RE5+O{`gXlKDt_8$wOeLw=`43C|nm}P2#*BWNnw)-$*YcfLTsM3!9JT2=XnhUSI=MXQsZyk;1oQ;-=L;lv`f;+e$ZB z*x9W?U;*WEf4;a)tcd2P)a zy4X1NrL9e=$*lMw=1R~hsT4&P{!8!`i8VbrXbk$TpfG~=BoZ|`KmY5h3?Lb+FE3ua zfbT>K5#W9f!W_Jk4m5swqrgJ;RB-j)VwIJOE)u>>t4G0{Id@fgyS6F?I;=71<6!2S zo*0sDplMjJxLXYJrM Date: Fri, 24 Feb 2023 12:04:45 -0600 Subject: [PATCH 02/14] Update CRDs Signed-off-by: Chris Doherty --- design/14_tinkerbell_crd_refactor/proposal.md | 215 +++++++++++------- 1 file changed, 129 insertions(+), 86 deletions(-) diff --git a/design/14_tinkerbell_crd_refactor/proposal.md b/design/14_tinkerbell_crd_refactor/proposal.md index 0f0ddbb..5f58ae9 100644 --- a/design/14_tinkerbell_crd_refactor/proposal.md +++ b/design/14_tinkerbell_crd_refactor/proposal.md @@ -82,90 +82,112 @@ The new set of CRDs will be defined as part of a `v1alpha2` API. ```go // Hardware is a logical representation of a machine that can execute Workflows. type Hardware struct { - HardwareSpec + metav1.TypeMeta `json:",inline"` + metav1.ObjectMeta `json:"metadata,omitempty"` + + HardwareSpec `json:"spec,omitempty"` } type HardwareSpec struct { // NetworkInterfaces defines the desired DHCP and netboot configuration for a network interface. // It is necessary to specify at least one NetworkInterface. - NetworkInterfaces NetworkInterfaces + //+kubebuilder:validation:MinItems=1 + NetworkInterfaces NetworkInterfaces `json:"networkInterfaces,omitempty"` - // IPXE provides iPXE script override fields. - // Optional. - IPXE IPXE + // IPXE provides iPXE script override fields. This is useful for debugging or netboot + // customization. + //+optional. + IPXE IPXE `json:"ipxe,omitempty"` // OSIE describes the Operating System Installation Environment to be netbooted. - OSIE OSIE + OSIE OSIE `json:"osie,omitempty"` // Instance describes instance specific data that is generally unused by Tinkerbell core. - Instance Instance + //+optional + Instance Instance `json:"instance,omitempty"` // StorageDevices is a list of storage devices that will be available in the OSIE. - // Optional. - StorageDevices []StorageDevice + //+optional. + StorageDevices []StorageDevice `json:"storageDevices,omitempty"` // BMCRef references a Rufio Machine object. It exists in the current API and will not be changed // with this proposal. - BMCRef LocalObjectReference + //+optional. + BMCRef LocalObjectReference `json:"bmcRef,omitempty"` } // NetworkInterface is the desired configuration for a particular network interface. type NetworkInterface struct { // DHCP is the basic network information for serving DHCP requests. - DHCP DHCP + DHCP DHCP `json:"dhcp,omitempty"` // DisableDHCP disables DHCP for this interface. Implies DisableNetboot. - // Default false. - DisableDHCP bool + //+kubebuilder:default=false + DisableDHCP bool `json:"disableDhcp,omitempty"` // DisableNetboot disables netbooting for this interface. The interface will still receive // network information speified on by DHCP. - // Default false. - DisableNetboot bool + //+kubebuilder:default=false + DisableNetboot bool `json:"disableNetboot,omitempty"` } // DHCP describes basic network configuration to be served in DHCP offers. type DHCP struct { - IP string - Netmask string - - // Optional. - Gateway string - - // Optional. - Hostname string - - // Optional. - VLANID int - - // Optional. - Nameservers []string - - // Optional. - Timeservers []string - - // Defaults to max allowed DHCP lease time. - LeaseTime int32 + // IP is an IPv4 address. + //+kubebuilder:validation:Pattern="(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)(\.(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)){3}" + IP string `json:"ip,omitempty"` + + // Netmask is an IPv4 netmask. + //+kubebuilder+validation:Pattern="^(255)\.(0|128|192|224|240|248|252|254|255)\.(0|128|192|224|240|248|252|254|255)\.(0|128|192|224|240|248|252|254|255)" + Netmask string `json:"netmask,omitempty"` + + // Gateway is the default gateway. + //+kubebuilder:validation:Pattern=(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)(\.(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)){3} + //+optional + Gateway string `json:"gateway,omitempty"` + + //+kubebuilder:validation:Pattern="^(([a-zA-Z0-9]|[a-zA-Z0-9][a-zA-Z0-9\-]*[a-zA-Z0-9])\.)*([A-Za-z0-9]|[A-Za-z0-9][A-Za-z0-9\-]*[A-Za-z0-9])$" + //+optional + Hostname string `json:"hostname,omitempty"` + + // VLANID is a VLAN ID between 0 and 4096. + //+kubebuilder:validation:Pattern="^(([0-9][0-9]{0,2}|[1-3][0-9][0-9][0-9]|40([0-8][0-9]|9[0-6]))(,[1-9][0-9]{0,2}|[1-3][0-9][0-9][0-9]|40([0-8][0-9]|9[0-6]))*)$" + //+optional + VLANID string `json:"vlanId,omitempty"` + + //+optional + Nameservers []Nameserver `json:"nameservers,omitempty"` + + //+optional + Timeservers []Timeserver `json:"timeservers,omitempty"` + + // 24h default + //+kubebuilder:default=86400 + //+kubebuilder:validation:Minimum=0 + LeaseTime int32 `json:"leaseTime"` } // OSIE describes an OSIE to be used with a Hardware. The environment data // is dependent on the OSIE being used and should be updated with the OSIE reference object. type Netboot struct { // OSIERef is a reference to an OSIE object. - OSIERef LocalObjectReference + OSIERef LocalObjectReference `json:"osieRef,omitempty"` // KernelParams passed to the kernel when launching the OSIE. Parameters are joined with a // space. - // Optional. - KernelParams []KernelParam + //+optional + KernelParams []string `json:"kernelParams,omitempty"` } +// IPXE describes overrides for IPXE scripts. At least 1 of Inline or URL must be specified. type IPXE struct { // Inline is an inline iPXE script that will be served as specified on this property. - Inline *string + //+optional + Inline *string `json:"inline,omitempty"` // URL is a URL to an hosted iPXE script. - URL *string + //+optional + URL *string `json:"url,omitempty"` } // Instance describes instance specific data. Instance specific data is typically dependent on the @@ -173,23 +195,33 @@ type IPXE struct { // service such as Tinkerbell's Hegel. The core Tinkerbell stack does not leverage this data. type Instance struct { // Userdata is data with a structure understood by the producer and consumer of the data. - Userdata string + //+optional + Userdata string `json:"userdata,omitempty"` // Vendordata is data with a structure understood by the producer and consumer of the data. - Vendordata string + //+optional + Vendordata string `json:"vendordata,omitempty"` } // NetworkInterfaces maps a MAC address to a NetworkInterface. type NetworkInterfaces map[MAC]NetworkInterface // MAC is a Media Access Control address. +//+kubebuilder:validation:Pattern="^([0-9A-Fa-f]{2}[:-]){5}([0-9A-Fa-f]{2})$" type MAC string +// Nameserver is an IP or hostname. +//+kubebuilder:validation:Pattern="^(([a-zA-Z0-9]|[a-zA-Z0-9][a-zA-Z0-9\-]*[a-zA-Z0-9])\.)*([A-Za-z0-9]|[A-Za-z0-9][A-Za-z0-9\-]*[A-Za-z0-9])$|^(([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])\.){3}([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])$" +type Nameserver string + +// Timeserver is an IP or hostname. +//+kubebuilder:validation:Pattern="^(([a-zA-Z0-9]|[a-zA-Z0-9][a-zA-Z0-9\-]*[a-zA-Z0-9])\.)*([A-Za-z0-9]|[A-Za-z0-9][A-Za-z0-9\-]*[A-Za-z0-9])$|^(([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])\.){3}([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])$" +type Timeserver string + // StorageDevice describes a storage device path that will be present in the OSIE. +// StorageDevices must be valid linux paths. +//+kubebuilder:validation:Pattern="^(/[^/ ]*)+/?$" type StorageDevice string - -// KernelParam defines an atomic kernel parameter that will be passed to the OSIE. -type KernelParam string ``` #### `OSIE` @@ -200,15 +232,18 @@ type KernelParam string // OSIE describes and Operating System Initialization Environment. It is used by Tinkerbell // to provision machines and should launch the Tink Worker component. type OSIE struct { - Spec OSIESpec + metav1.TypeMeta `json:",inline"` + metav1.ObjectMeta `json:"metadata,omitempty"` + + Spec OSIESpec `json:"spec,omitempty"` } type OSIESpec struct { // KernelURL is a URL to a kernel image. - KernelURL string + KernelURL string `json:"kernelUrl,omitempty"` // InitrdURL is a URL to an initrd image. - InitrdURL string + InitrdURL string `json:"initrdUrl,omitempty"` } ``` @@ -219,53 +254,59 @@ type OSIESpec struct { // prior to execution where it is exposed to Hardware and user defined data. All fields within // TemplateSpec may contain template values. See https://pkg.go.dev/text/template for more details. type Template struct { - Spec TemplateSpec + metav1.TypeMeta `json:",inline"` + metav1.ObjectMeta `json:"metadata,omitempty"` + + Spec TemplateSpec `json:"spec,omitempty"` } type TemplateSpec struct { // Actions defines the set of actions to be run on a target machine. Actions are run sequentially // in the order they are specified. At least 1 action must be specified. Names of actions // must be unique within a Template. - Actions []Action + //kubebuilder:validation:MinItems=1 + Actions []Action `json:"actions,omitempty"` // Volumes to be mounted on all actions. If an action specifies the same volume it will take // precedence. - // Optional. - Volumes []Volume + //+optional + Volumes []Volume `json:"volumes,omitempty"` // Env defines environment variables to be available in all actions. If an action specifies // the same environment variable it will take precedence. - // Optional. - Env map[string]string + //+optional + Env map[string]string `json:"env,omitempty"` } // Action defines an individual action to be run on a target machine. type Action struct { // Name is a unique name for the action. - Name ActionName + Name string `json:"name,omitempty"` // Image is an OCI image. - Image string + Image string `json:"image,omitempty"` - // Command defines the command to use when launching the image. - // Optional. - Command string + // Cmd defines the command to use when launching the image. + //+optional + Cmd string `json:"cmd,omitempty"` // Args are a set of arguments to be passed to the container on launch. - Args []string + //+optional + Args []string `json:"args,omitempty"` // Env defines environment variables used when launching the container. - // Optional. - Env map[string]string + //+optional + Env map[string]string `json:"env,omitempty"` - // Volumes defines the volumes to mount into the container. - // Optional. - Volumes []Volume + // Volumes defines the volumes to mount into the container. + //+optional + Volumes []Volume `json:"volumes,omitempty"` // NetworkNamespace defines the network namespace to run the container in. This enables access // to the host network namespace. // See https://man7.org/linux/man-pages/man7/namespaces.7.html. - NetworkNamespace string + //+optional + NetworkNamespace string `json:"networkNamespace,omitempty"` } // Volume is a specification for mounting a volume in an action. Volumes take the form @@ -283,8 +324,6 @@ type Action struct { // See https://docs.docker.com/storage/volumes/ for additional details type Volume string -// ActionName is unique name within the context of a workflow. -type ActionName string ``` #### `Workflow` @@ -293,68 +332,72 @@ type ActionName string // Workflow describes a set of actions to be run on a specific Hardware. Workflows execute // once and should be considered ephemeral. type Workflow struct { - Spec WorkflowSpec - Status WorkflowStatus + metav1.TypeMeta `json:",inline"` + metav1.ObjectMeta `json:"metadata,omitempty"` + + Spec WorkflowSpec `json:"spec,omitempty"` + Status WorkflowStatus `json:"status,omitempty"` } type WorkflowSpec struct { // HardwareRef is a reference to a Hardware resource this workflow will execute on. // If no namespace is specified the Workflow's namespace is assumed. - HardwareRef LocalObjectReference + HardwareRef LocalObjectReference `json:"hardwareRef,omitempty"` // TemplateRef is a reference to a Template resource used to render workflow actions. // If no namespace is specified the Workflow's namespace is assumed. - TemplateRef LocalObjectReference + TemplateRef LocalObjectReference `json:"templateRef,omitempty"` // TemplateData is user defined data that is injected during template rendering. The data // structure should be marshalable. - // Optional. - TemplateData map[string]any + //+optional + TemplateData map[string]any `json:"templateData,omitempty"` // Timeout defines the time the workflow has to complete. The timer begins when the first action // is requested. When set to 0, no timeout is applied. - // Default 0. - Timeout int32 + //+kubebuilder:default=0 + //+kubebuilder:validation:Minimum=0 + Timeout time.Duration `json:"timeout,omitempty"` } type WorkflowStatus struct { // Actions is the list of rendered actions and their status. - Actions RenderedActions + Actions RenderedActions `json:"actions,omitempty"` // StartedAt is the time the first action was requested. Nil indicates the workflow has not // started. - StartedAt *metav1.Time + StartedAt *metav1.Time `json:"startedAt,omitempty"` // State describes the current state of the workflow. This fields represents a summation of // action states. Specifically, if all actions succeeded, the workflow will succeed. If one // action fails, the workflow fails irrespective of previous action status'. - State State + State State `json:"state,omitempty"` // Reason describes the reason for failure. It is only relevant when Result is ResultFailed. // It is propogated from the failed action. - Reason Reason + Reason Reason `json:"reason,omitempty"` } // RenderedActions is a map of action name to RenderedAction. -type RenderedActions map[ActionName]ActionStatus +type RenderedActions map[string]ActionStatus // ActionStatus describes status information about an action. type ActionStatus struct { // Rendered is the rendered action. - Rendered Action + Rendered Action `json:"rendered,omitempty"` // StartedAt is the time the action was requested. Nil indicates the action has not started. - StartedAt *metav1.Time + StartedAt *metav1.Time `json:"startedAt,omitempty"` // State describes the current state of the action. - State State + State State `json:"state,omitempty"` // Reason describes the reason for failure. It is only relevant when Result is ResultFailed. - Reason Reason + Reason Reason `json:"reason,omitempty"` // Message is a freeform user friendly message describing the specific issue that caused the // failure. It is only relevant when Result is ResultFailed. - Message string + Message string `json:"message,omitempty"` } // State describes the point in time state of a workflow or action. From 84837a23a75a8f1d72ad2abc66f2ec7e1641c4f9 Mon Sep 17 00:00:00 2001 From: Chris Doherty Date: Fri, 24 Feb 2023 17:17:04 -0600 Subject: [PATCH 03/14] Add rationale for not using conditions Signed-off-by: Chris Doherty --- design/14_tinkerbell_crd_refactor/proposal.md | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/design/14_tinkerbell_crd_refactor/proposal.md b/design/14_tinkerbell_crd_refactor/proposal.md index 5f58ae9..673e5cf 100644 --- a/design/14_tinkerbell_crd_refactor/proposal.md +++ b/design/14_tinkerbell_crd_refactor/proposal.md @@ -20,6 +20,7 @@ - [Migrating from v1alpha1 to v1alpha2](#migrating-from-v1alpha1-to-v1alpha2) - [Rationale](#rationale) - [Comparison with existing resources](#comparison-with-existing-resources) + - [Use of explicit `State`, `Reason` and `Message` fields vs conditions](#use-of-explicit-state-reason-and-message-fields-vs-conditions) - [Implementation Plan](#implementation-plan) - [Future Work](#future-work) @@ -486,6 +487,12 @@ Reasons for failures have been introduced as a core concept, via the `Reason` fi The `Workflow` provides a `TemplateData` field that can be used to inject arbitrary data into templates. This facilitates users wanting to model variable data on `Template`s that has per-`Workflow` values. +### Use of explicit `State`, `Reason` and `Message` fields vs conditions + +Workflows operate as a state machine (detailed in [Workflow state transition](#workflow-state-transition)). [Conditions represent observations](https://github.com/kubernetes/community/blob/master/contributors/devel/sig-architecture/api-conventions.md#typical-status-properties) about a resource and are not state machines in and of themeselves. Conditions should generally be complimentary to existing resource information, not replace it. Conditions in the Kubernetes API have a [well defined set of fields](https://github.com/kubernetes/community/blob/4c9ef2d/contributors/devel/sig-architecture/api-conventions.md#typical-status-properties). Some tools such as [Cluster API](https://github.com/kubernetes-sigs/cluster-api/blob/main/docs/proposals/20200506-conditions.md) break away by defining their own condition expectations. + +For these reasons, we think it would be more appropriate to have explicit fields to represent state and consider a separate conditions proposal that compliments the CRDs. + ## Implementation Plan 1. All services that have unreleased changes will be released under a new version. This provides a baseline of functional services that can be used in the Tinkerbell Helm charts. From 9c9729ec0890e2427a80d87f7a9d3a6018d0e334 Mon Sep 17 00:00:00 2001 From: Chris Doherty Date: Mon, 27 Feb 2023 07:59:42 -0600 Subject: [PATCH 04/14] Tweak dir layout and add action states Signed-off-by: Chris Doherty --- .../workflow_state.puml | 13 --- .../workflow_state_machine.png | Bin 14834 -> 0 bytes ...md => 20230222_tinkerbell_crd_refactor.md} | 100 ++++++++++-------- .../tinkerbell_crd_refactor/action_state.puml | 14 +++ .../action_state_machine.png | Bin 0 -> 13184 bytes .../workflow_state.puml | 14 +++ .../workflow_state_machine.png | Bin 0 -> 16265 bytes 7 files changed, 82 insertions(+), 59 deletions(-) delete mode 100644 design/14_tinkerbell_crd_refactor/workflow_state.puml delete mode 100644 design/14_tinkerbell_crd_refactor/workflow_state_machine.png rename design/{14_tinkerbell_crd_refactor/proposal.md => 20230222_tinkerbell_crd_refactor.md} (86%) create mode 100644 design/images/tinkerbell_crd_refactor/action_state.puml create mode 100644 design/images/tinkerbell_crd_refactor/action_state_machine.png create mode 100644 design/images/tinkerbell_crd_refactor/workflow_state.puml create mode 100644 design/images/tinkerbell_crd_refactor/workflow_state_machine.png diff --git a/design/14_tinkerbell_crd_refactor/workflow_state.puml b/design/14_tinkerbell_crd_refactor/workflow_state.puml deleted file mode 100644 index 3b8c2f5..0000000 --- a/design/14_tinkerbell_crd_refactor/workflow_state.puml +++ /dev/null @@ -1,13 +0,0 @@ -@startuml Workflow State Machine - -[*] --> Pending : Workflow reconciled -Pending --> Running : First action requested -Running --> Succeeded -Running --> Failed -Succeeded --> [*] -Failed --> [*] - -Succeeded : All actions completed\nsuccessfully -Failed : An action failed - -@enduml \ No newline at end of file diff --git a/design/14_tinkerbell_crd_refactor/workflow_state_machine.png b/design/14_tinkerbell_crd_refactor/workflow_state_machine.png deleted file mode 100644 index 2470709d63cc8dc17030b66532a2c0b1aef03c4e..0000000000000000000000000000000000000000 GIT binary patch literal 0 HcmV?d00001 literal 14834 zcmd6ObyQUSyYI}%AP#~c3`j{SoeCmI3`k2$gLDZ}N{5UhB@zZGqJT6AN`tf#Vjx{2 zEg{_<_Zi>cIqRHz&RX}Zd;htAyz7-cdw=&gpXU?Lj?mIjBE!&Q5C{aBin6>80)f*(#}JYUv->f}*UyzNJySH8(V_I`Lc@b&cepv>Y91wwX1e(6NUYHv23S3Jk& zYQRkK`BZ_tk(-gMpQY%9^K`U^^67EMcE{6iwbS)Aqio}O|Ew^FH4|5!N4>t%Oc8ms zK<7-HJZasR6`{<2W42BU!_KM^P5dIa991(}C4v;r^pKlxW0&L~^qKr3#CEkVQbtw2 zMMvvq-vG)hTOChalb(F8rb=Tf^6^_16^q#P16jYJuoWA$?BbIjUAK{n;;j@Im%OQD zffpWMUM)IYj-3?Eq{#ijGbbnQoG27`E=_^Kr6FOz8KJip60&$sBMP63d z=jXy}e?x4~@aGyQ-)|xf3<)IMY?T*vAFcXzUN`Fqqw-UZ4(l}F6{Wzwci?FjrlyR+ zqs(H|K!)WA3uIbtrvLw=}Z)JElSBoq<&94N|6or>u$s& zkKMj=$H)}g`n*lMzBI!8t+*X-VB}DH(Vlk6V(0j&vjxB40@0JFPyM@-8XM(ln6S8K z^)H02XH)QF4PnJI*$AsO(+{^Zlx}>iqaZOcu|xH6 z42vqejLb-j3~92Ie|(byngY!hdQzC;D3VriDi-EJ`)(TkSeQ2Q!9K(B=-y;bIvH>5R#Dnt=k#hQotBGUi&o!7^v3_LPtB>kd`X6r_D9(;GLA_NQec zpT$V;lJrf^4-_}eCrSBBc+B7#w%K`Dq@`>1-sdsV!$i<T9t{J%cqhH+ug zDLEtA+G#U04yDFXC)~Xy{MKj0MgqDDEU$`8FiZNR=;XNdu%X;Vt*z?u$W;B9$g|(7 zSDG2Z$eDJhWsbP55IZcl6(5~(KROKLyRM@%zOXPV&(GiLc4$QE>*>ilYfA5&`?)aPmVC6{rOB*o zJ(}`T>-i%NE%^V~U$v1q0%E(1mAv8cT)56#~1>OaG-XKAp8 zdCv9P+SxsvrWY3%XA|b)Qoh6AaD1cln&83eH^CQ*tQKiHKa3M*uH)xjaK~SSM@Cv- z#T&jxx!HQpv|Fx>H&JnFG|t0`dF0O`{rLXC@8342dFDX}rG{tR?+?Vt92%ByL>lRC zuPnf5M8(oIk-N7Qm#-rIH|GJksnLkvrA zD&Bu@se{pqM7d{am5>7U*E~;=`^`3KFby&^GyvxH_#TFaOkrW*c}vV ze&SIle>2k1aJZzusP&xFPg9H%isUGCq_D z#_ZOIoOcd(HyU0WrEe_^ZoT#%eoXV!j`D0p?_jyBxuIdWkuF(5{Gt`{ZE5dCMjh0# zV@-KsVM?E)nLJI9m3wY~Iy2r}4YL{wC1WUa>Ras6lsU++965WN!sHJ`HLL!X;GY%l z6B98q0-3fKJ4_Al7<-95dGdsX`kaKgIKN>59>b!ajl}Fd7KB|orntO|C=H(EDta#C`hP~AF>fVc*q+6y7YgW zMZy}0rd9^Lr)vvp;9 z`U3@dG)2h7?Ce7a+NBaojPkW>4f1{OZWvvL>?UnwWJKOmkJ^^s_nz%OLFGVF;08&T z_7UAh;;sE3udlGLt_);cd#;_PSZD}82&QLdN?f^eWj`81rK&0>G6ci2DW|6wr0CqrRbY9kgDgSAri|XKYskk zhk?Y-e*XM>n zuv4mhR@gKdvADBwds|DNo0|oUiq3I!|JmPJb7UCXTdEr=b?O^$iaf=5x&Em#I#%NL z)c1rCQrf$F3+0Z9=lS{hg@j`3E^8svdtt67e!MRWd}m%4J&QOBhP#a^=;(acXS7Ro zxC8~m(1x@rsBrD{^mIlE59NG5Em{%`X8cPAN}0^G%8Q?kO)g))v$Ior=QmZ76%YOl z8wbbKRBIy5i4H<b}>{TR%k*OjV3YHvT?^VXox zAmEbg@cxkZ2)2t|@zL`7dY(6WeyHmCYswr%Dy_V7lnj!b@p6Gv-#ej@Q26IMmI#IYrxqW%;W713R^D}k3ma)dlm8@N|y zXt%rmM_O7M;+i4p2y737ud^`9?ac_zq?!K03CM{rFFEsYZ&u~xB>S3;Jk zcp0zx7u16!dvyR}PMkP#_UzehRUJg{r%=c5V;gdTFV#5p2?z=bwlWe9*b7zU zYY9)krJ5~(B9ymb(_o!%J*deh$4dw9vanL9=)y`!9ZgbC7RRzSx^@qAck# zL&4Vd@#7m!QGS!MfT;v~kKmoGZp)*xYC+2%j~9t+ zA$1YLzE^SM_4kg2EB@+a58}LZsn-p$G*ZXh6?C*WGM9&+_#9*H2be||fyMFVA-caX zT)K7HZ{1;Nas%yl^CtJ~0t9+$b1XG=N|`w?+M@Dm82Rw<@c7&9Z69e;*rm4Ah)j*w zgs8T&Hk~ZtIXlyl*12%Q91;B7%r6I4^R0e9K!DpG9wN(^wam%8GSwUMR5R$M1AGAS z3q>*g9kLwyB)v$>(9jJrCT~v}8JS;SD8tkfFDa-!T(@ipwZyg}$WNT0ELMtU{L^2^ zu47#8QUf89hTm8j({B0=N8b0|958iRSlDr*b%_09j@=FriV1^K%SJT(s`TXLiMk|caMFD==6bnqrbGg`6&8P@bkr93&u^n78|oYe-9idD=Vv* zn8C-DAb=FGCzk)7tJT$2*fKnZNJxTQTwJ3OgbwMn+%OuAyV;vvYGXWMq1V`<&;`+uGZ&{f5-Z z1wsEgze(ncFjPW90;I+1n-G{5_zE0*c+e#z(Tw7!>F6SI$=bNFIL9Hj{vun_-1aht zBS`tG>gtm@=_utPZZ0l7Norc!+S=OEP?i0paM0G`5bP3CT7kx-IO@MQ z<5w3nDk8!@0L()}OUn%0DC4=AV{e|(bAcPcxC`s=qe)+ixeix)I?>WT2kLdxHY9+U zjQ#=o*U(VRSt*{=@Bf`_urfL6>FdwhrG7YN@XjEz`{?_nlZt5D4! zL94`&5p7y1hS+<49X))Ad4XDqfBpL4$CagUIb1bs5MO*!(%t!f;cQjRCymz<`G$pZ zzl%Yxa2cw!L?!mN5?wK}{?hW!!D0>q^HbE%lACL$q7HYz@mPF+A*@4Ic4zuWIL@@I zG`E3$lSa_&(H?82=4*+Nw~}03EpD%keue>?vG(nbRoI1`APj`SsPLpqb>03dX(nK- zNQRs|RQdsJVrU#o6@e9Z0YM>AR)Z7uN59Dy%aRx&eE#}i{&k3UWJrFra@V+oVkymY zh^W@CumEW&IFXd%+1Xn&9ap(n#Kq~}P9)t*M>Vbz*om^zvh^ka*WRDy;o&5KnAmtq zBTLQQeR4-GHu}1qDF*ic_yT$?%PgwGt)s-O#Nn~Wo4uW#KQ5ENd7inrkByDVAvraZ z})P|^n(Wv z!q;zhW!~&Xm0S?cv~jHRr_Xl7Iu($16U!LzIr~?2V37pb+C*@Bx zDZR7LgTm*}4gO*9CVF}yG=g3T?LheZ*m3gx{k68r1m(f}yu8NORBxw&l1xeZERTYw zz*b0(p@?C1doO+aH_2_B%X2|<#mcla_En_=Nmf=Qk`FxwLjw{|!M8c=zr?D|1q{eq zHM7hl3l-i(fJ7tS!@f%1yD-Ey>AhPGp=2On9fry5kL_;Eu`zA^womfSRC&swlcP2n9}OgZ5X|v>V4_h@J;}cPwrEUc+K}yUW=v| z#Uv0YqLs~;?urdGDGemSJWTenimaR-#~}%!#e1jDO19sq&C5F}P*0SLc*#2+YGh{S zV#9$Tn`aK%FEk@(uF^I(&P})t62OW{n!eNibMw0$KUIz;oH4pUTxM1WEuO<@UE0{g z{U{=&wrtUXfq`$&1r3@D34azi{gde)5K z4_t%3qod=`pFc&ImhaymMMPLqYJ-gUEIyuXKI6HKDkeN8=Cg;2%85u2h? zk3Sb2ySZpcfn$Vpck9~Q+lP$+CZ=?4f?gDo=H}+=^2BAM@en*78U#`TlpuwkjelT~ zW@l$NA_$0r-HxSCpHfk`*Z6x6d0P;;R`UOZtvLQywzBcRu@z=4ZuSQtcA)6b`HY@C zd**St?^~+AaafFqK^!orYqba`LK#!@6X@Zejtw6!U*Sli2YiBFyVmHiUZ z#UpxA9sz+L9)%KCjU<|yZGLTDAiB-RKyss_U=Tedy6iSu59{=a8Wb7lo?NV{+iW*C z6*tS}%Ra-^zL_(Eh&wADKr|9J=KA{1$Y|k4Y6F|rpPau&FX8}H%jhjh^_^)ArrO*F zE8_actAB6(%Sl)`IEXR*AMRwmU;;|g>0I%F2VjoDJ`&*Z{aAl))EC#R>M%NvB( z1C&&y>)qCiiX5tNKQjxE-0f7@LR7}8@vui-{mx9K0RS+9nQ zIR9L4jG!9l1@&8`sY&Gih$N=o0|dnCnwlm9hZoC6aOA(gK`g@n%N;sW4c?pC+ShpMV;#c$rcArW7p zx}m73*p?(E9C=wyL!-GdjQK7X4G@|$`rRi-0qGkD+FBICK89Nc8yc)Z_e~Bu^aVmF zlE_JDaP~SKdzZ@;k=||0MPn{!E$d8kNy7# zaxg;BQk~|HF>-5){kPh115M=-V0Y~_110sc2Djtaa+TYC#sq)t^)27cyO_8+a zw+23SsiP^Jf%gFC%Jo4rUUmZ&9#7bt7jQ_5dqvm+;HQ5N;~fGyI6hW>^WVq;oX{Ct zknJ7#dG(NyH2k@FdB2Sav;tzA+|{dCx+5Uy5ZDPPKIb0D5)iz&=ilJ$?504#^u3*f zu*yU0=No6dp!o>74MK^hpGq2)1q46}^Ok0`4<{alNid_jOB zUewpux3_Dpxzc(ffq+W?299xFHqalaIn5u};3z+T|FqJ!V#0Qaef~U4Cx>slnk+xi zz@V?BL?nq6fGR^tgOrq1S63H*6$3D%0-TW%T^ppd?+@$hf zac<5@DrP(&AOO%x(;CQFcVgq>CdbAs_3A3$_BaX(zJ~!NF+syLC6$y%`rFT|?%zjw zdv8@&OD3_yWlZ(^wY z1$7kAK$hde^tAn*8!LqkK>Z~HbrHV0pBfsbzb9Bn`jg)=QaMt9yv73y$gUJ!(vdu{ ztp)owMqXC-p6wBdq4_Das;a8bCE#;cK#hWU=LH&tiA*f?3)Gau2*|;uDY>~{M@E7M zifq4Leg#JrLhEV?%#sQc?BOgATpN<85 zG_$z)<2ktw^7qKnZ#tQ~)%Tf^v(_*MBoZ0GOoz2tdJr0Fykls=A@@ko=5aY8cN#0J zs1y##prj=(yE*x=o(9?Fs=hwW&xT6}O7%*doSer)Qd-_=0YP}+L)4NrUZ)omAq0X9 zFEdRXhGpZ|$<58pDWR&() zb7hZod9Z{_T(6!yIqmK3T{Vlgkpx@>S!W^{B;2M{{kQHB#s1Yuko4k2^g+-#JZ0Kj zTU$GS;yOEsZ&HBCgjR8h3YgR$}lQ+ghWgeP$EvdOY>H9gIijmaar4PmHm~dY@yM?X|U#y~?(> z8NtE9+di`%5d~`hZd81`(pGXD6N4>qghRd88=2E9*_ajKu`PG@O$+Vv4saMX6G{;;yERZGbGQ;i5K&i}XzA8Gb#ac4Jn!Cs>r-WQ@tisbdNH`ud9U@&z#| zg&0EsQhGNs9b#Td`d*NcIRb6BD3|GK)9EP3>+h{}Tt;f0WscDSfhj81Z~Z&P00#4> z>4(ak2Z5Cjrx0K>qohx}-)265G%)Jk&!B*x^~4$%l1pyb`^Xum!Xs|joeEI}ok3rOtnv#aHHwCygE@1m z)2AWI#lE3#^E7Po6Dmga&o$mUvv^?1fT_4W`IY%_`MA8sw|IV#n?NZC{NX&KiQs|0tW6yS!d3u93*ID#yTd zucf`;3Tx%U5kkW+Il$G9e`Vi#k;RKI=jfFi`6X7p+ubh2)&?EjO+PwV3hIlR{`+h} znHgv4H~{)=XT0H6O^U!8nOkbe`hfx16+f(gwzjJ4mLcz(>(m2*&@!qof64om1CSM) zJEUM^laA+R#>SZADs3#J7z`NoY=l+m&qC$B>H9+Pm=1zmepOgn>`<~RhHFP47jYo2 zJD$?q1ydvHk5I0L~nxp46!sJz+b%%(6gB1q@s4iEi0EEI^_yvNGP?gGOEoTZf*TfAr`PLFTD};QZyw zmlt^UMgs`DuSIU2o{w()eAXW#mF@0n_vf2yYYngIB48^rNe5JTY{jd~-2SZ$N8@on zNQ|&$fyT&5RKYTnV=c&J#MwhT9*QiFs+8G>Nu&E!H7N#$naesTaN1K-Q%RD(I%O%^ zrM>0jz){y$S4)Tl6Mg1-^IyGU3~-!eNsRVv1Ix?IOrWYmB2Nx&7(tNdppc^n*icrI zpCgscHDI!2g%|e^4i=P< z!Ng=w4irW;gLbb!bwr_1W`P@eImLvspn`{nig$ia`Q0y|0dpn$-uSo`QCVfBm?|)V zd(v{JFLhJ>m5kah5QtQgEx}2V{PvHll5~zBGR62j5&5j0)9`|4GvW^>*TuRUR3}_3 zCpRZ&7D&-hU1I!aS3;wk_N8?h7x8uKXDKd2+O3(0O^L$XVdm27^`MtD$-LD_lDw&d zz6C`QLPAv;u+#r8G?% zj((V)cyPNL%#~>;LTmB5Mr!lSt5pzOYxY;a8LD-f zNBxK1A+B=%fm)Do0it7>l*DYtvzvUlJCZa5V*4}t_zX!xVvt;%W;=jPj5D;dyeTX{ z4-#E>-O<3Eoz=X0E^Cfv;+1${od8ONN~HMZ z%gV>5QhDxC^hj_82DdrH1?T*o9e6+YrAzvcQetnA2{`FGWzc@2OG6B-7}|_1TjP+a z!uJPL*0D5COU0h%63sC}@a@t%NqFh@=cEV9{^H2)-D!iyJNc7cI`8DdNh zam2YR4h~SIsNmcz#>NDG3DDyE4g&;jGnI`~5 zuwV1Ek+ZWOTeK5s8>$_gHN3efKJ8gt<|xKgQzo?(bl*d&t~?|i=a;H7WU6>su6 zjl@gp>1p)$U)IgL%<$mOrN%`|Ddn48yz#F$&WzN_NM>)u)qb=u`TDWjy4%ncx$F1( z#l77HpZjx2dyE17LtSqRoq?#A3mfM=b?d*z@xDR)K_(&k*iJVM);j6p@BN*|Vyh<1 zxy<(AJQ5|L%27sf;}hr-31% z+a{q{e5eFDCn!@;2lbJTnmRo(_@54?|9gi}PrW25m&zKN?0keHs|l0f=II$ySh6a2 zp6D~T8R;8WHLUtSQ=JT){1;g)+b~ORg1xqO2Y6f-Ml0`DK>t)tY&TWoLTGbRFaNsJ z7NMSk<^NAY0Nk1&gl3@Z3i&ApNLT#ZM(`5H(Ng@#DS9Cq%rd$Wth z%%Vx8(L|D^EJZ28R!J3h)aR-(+KVdF)lmWesIzEt3I;hXndj?2ux74Xun7`RBgaCN?Hb z4+d040zkZ8jmhKEos7!iTd#Xpu3&PMQR*Tb0tALP7qxqgj?NR_{X<;Q4F zV?Og*(={XfM)e!zUo6Zj$mWwDEWh(qMu>Vn5Q^m*qzc9q7^Cs<=*w{mzQL7rmn)eu zeoF`Ldz;Q&xGmE&ZxI8mV(lqU=sC6gA@*u@JR!$<5s?qc2QsnRlxHVq4^lPMn|w*AoM&GGkq@yYzuu--<1ES zPwVK!TR|s}MHHpkwzz0t(w?*L|H(57GIpnhG{Lr_;uR5jh7l6=UK&=2)7m;RP9GL` zA8!OmvHqIG8q0F!cWAA}(}fsI7HzMRlLP!;IiyojPyl+YT~d~G0U)D8XZndodGw3v z4=jB3m3Wl_a4l}#x&@Bv0(Ai#$f0VFuwU)8w~7vgPoDc49sj2HVoR9Icp5LF^&DSl zFLy8080ImVaRCFkYBDcY99)SII3Z0dg#Pg11DA>JZL(yPFPep1H0VQ5085Ve@6g1= zL|s?fy6+K$Zm}jhSp3s1M{eAR8IZHtBY<7Bb*BT@JCwY9K8%I}0z60z!CmwO)ekuI zlbTuO{0g6yaY%*Wf!wDuh9WfUv2VOAGBV~DPp^}G7+YEz0r2?xinrV)64us#Uc)s= zDd!cr>vi=Sl6doLea!u*9O|dlPyXRNQN<=8@UEm}T`Uqs*7W&vd|`z>-oO-9rB1@#DuG#rB%#yyklO!CR%u){cB{vuEAH z@zv`=>`-d$Q2T6zrN#S`C!JA~U%WWVkAqt4jy%Y80HoX$Yg0#QO^qmX8QNVaA8|C} z+70~$A-Ub#`OD_#SoLQMgxzATcqmkDP$aLYn5uGn@t61&U4)|=3>Q=T)m^xvvJ$qP z@Tk)pJk+nH`Py*`v&_=!XsY=;Nu<6>AHBxdqXyc?K(u>fEUF$f8k?|z>u0l!z@LyAF6IU~iCLpZNtE=<7qJM<<(%7ndw@T=Rxf%-PB7`WekZZf|<;`dH;Kyl? zrjMWkD`;$SFQH@vQR`=EEa;sjk>#mXM|sZvL>aOw6qyYb*+v#Q9o~oPr>e?I z8Z1`0dd?h~?vipoB(eyEaRp09n04)L&?*Q8gL>Dijy=)K(alB|aAhN$h@6qGuCC}K z5?7;cN6~YMuU>s8s64bmK|!D#AFAms;qstn!H$-Nr~aqx@`UP*Vmq?P)VtSu`ufD6 z3eY73bqf#XBCtNA<+h!Nw=89mkAisW?YOwP-Mzi1nfqqZx;fhqLNEnV(2X0WrbWfY zM$|E%Ku_1s7+>Pk#j8Jg`EnM7hbtvlgo$<2fUVBIDW<}v7@nr0@}+vq0ITkx2IgEL z^f%G<>+bgUnTy}qA9L~W=;Y`+s1Yuo7jo(agXL67RC-?C_pYw%BUSDsSPTCY%;>^` zl=q@3jY3qo0FDv;2%|-xdgtxV$h5TyTmY*D<^*>Sw1~>8s^Z8hq?Wc^S_D4_53Z0P z+Er96$G8*~kWyeRXw$Q@V18H21^D~X;g|w;=$#%9*1uZ$@6R!D!(a%kE*R7m67DW1 zA|@s#BI@5s6ZAr+qULE=jzRIzxA`)nzYlxQe_%LCFFTKtN5=+v(=0D97nDCnKiDJu z$a0jKl2Q+t^PM6M@D&S~dHg;zV1WYs>wO*$Gols&oGxM~2&5$+Kfn06IPN_mb~d)Z zkNPz$Kg^z|9!L@~vwHd%JVg=2?@hd>`OSF$0?H_Dh3LA~b7 zoZMW$h0?xf-L}deNH{ptP@NgBo!#SDk^sgo){PHD`FRwfvNQNn@!Pj=^Od5{@ZN-| z_t)bP1(~`4zYs0RW-P+U8--!uu{g-4oFP3|(%yEn);n1;u zEPQ<5%bIB;!o%yyi%EsxrnO8*-uLHJH8eJtmqB!#1T8|jhP9ct@G#R8l(Z@Y$j!Pm zXQcc@X8ZN4!l2L^I1Qmw6xRuN{dnw!!C>5urg=|;EpnUG&@w$j03VS>r z8=b_z@8G}{Z|LcXBP@g!Kg=db)&naFXERa-_CBY>Hv_b^w6!G9pqM;l=GEIP(j#e) z`k`V!H4e&q?(ZFa3_fl&3XMs?;esGsGCLh};oOJ1y1FSo<3s&HsRS+LZ<#G;s2|=@ zx^et?SY;`ihnKhT-PHW{%ESOV7TWzmD7qi%4-wK(Q_H(5oxN}k14VF9IyAv$0WFh( zUgZ9nX6&I5u$5a(7i8nDEs8cop*%5A>038G_hex8ro;!Z<6B~Se# zO8Vl(i~RigDJi&0ZFVd0i70DGCV(fWd1?*nXM9h>cVV??PlI^4MaPMcJ@@0wk9V?)DL8!QDyEVwu* zC^`|<{xvjXUXm!9o8OtjnA)%Te36k|ZCBQQVM9&6P#)2~EIoquw zFX*_@d?bLsqyTDKM$1xf{$RE5+O{`gXlKDt_8$wOeLw=`43C|nm}P2#*BWNnw)-$*YcfLTsM3!9JT2=XnhUSI=MXQsZyk;1oQ;-=L;lv`f;+e$ZB z*x9W?U;*WEf4;a)tcd2P)a zy4X1NrL9e=$*lMw=1R~hsT4&P{!8!`i8VbrXbk$TpfG~=BoZ|`KmY5h3?Lb+FE3ua zfbT>K5#W9f!W_Jk4m5swqrgJ;RB-j)VwIJOE)u>>t4G0{Id@fgyS6F?I;=71<6!2S zo*0sDplMjJxLXYJrM ActionStatePending +ActionStatePending --> ActionStateRunning +ActionStateRunning --> ActionStateSucceeded +ActionStateRunning --> ActionStateFailed +ActionStateSucceeded --> [*] +ActionStateFailed --> [*] + +ActionStatePending : Action waiting to start +ActionStateSucceeded : All actions completed\nsuccessfully +ActionStateFailed : An action failed + +@enduml \ No newline at end of file diff --git a/design/images/tinkerbell_crd_refactor/action_state_machine.png b/design/images/tinkerbell_crd_refactor/action_state_machine.png new file mode 100644 index 0000000000000000000000000000000000000000..a370c8c96361bc671dd90614a6b72e71252ea4ba GIT binary patch literal 13184 zcmdsei93}4`|mT834?57DTGw^Ewl(tgk<0MEN#|gO@t9r$dVAU?^{&%U1+gysq9Iz z?>lks>GS#i&iP&EoIl{4uFFL;^E}V{eZTMJ^?Kd+Jk?NBq{1GEI4_5{2lv<;Zjxkbz9T)flENv# zdBj~)ZpJNEwK6)tjCsGt{%Eqp=EJqb-izW*EN@Flu7`Ko%!>Rdoop`CV1DzYX7r&f z>!k)dU0xlAH*RDT@w$BB^E97^7%`SlfBATP2v{4EKay|E{wC6;`D-{%A%$u0-sMCN zs+5&c>5h|~>>?Ud9>Nhtdm0~%L%tjGhQ;81>lGYzv(I`$Wtx0ZuIZbWll-h5iqtOT z9F>RCB_$8)gf`!Z0uCkf@bo{IJq81pZ(^=5yzOXpLMaSp;&}p66doF~_uO*!9^2PB zX`|AR*1c|h(k6%ibCr9<@03#|sO%%@7nai-2E8z-!k#c3Gi+34hzx-~iQSNy}DKjNYb zQlg-s`uU2iDdL_*Agf&f!-P^Zx z0f#V%sPp1T&HP~5#q;Mwx*MYfjTeS0{2o0*6V9{mM{=tl@1=`8t;#1Xtmt?dg%4Wp ze0kpSgkYGgADZC)@{-T~%YJ<8!QN^o&p96CiEmq5n`i;Igam^;Dco7abHk1cCvxKh zC5{b;8j+x^B@N23=Itx_zC^;C|Oi}<@Y+Dx^8T;t0qAM)(k7}qCqr} zB84SM8Z4qkYKE0X;d?kF$cX?b57EQ@i%)1S4pTs!0=u($rz zEUi2l;gB_m;Y2UjS@)iHIf}+0%jd9Nj?6fu-sjDmH$076uIW|Ncv` z8v+`36PNS)HKPWhKMVF~$t5K8Iysr3k7{mie){w&&k3+qY-Ob+cGusZL=cmnmDTHF zZ!Z`G#yxRoZ)+Jh|KP!cS7@2tIk7lqQA-jmDHa*eN>6XfJo*;-ld;7v>B6I`A%_sX zM@huCKi{J9arccR={+UX@#DvN7>|Nc_UqljQhPXP z1_lNVMC1jmr_$lfnKLb|tzYVQD%|Fc)YLjA+vBN^OYt-;%A^qVmX>}ruFrn`{rfkU z-4*f+39lD3lw*>Sdb3q(a`FntpND1Thrm?ar%owlNg^2LoxQBB!oosD|Es>MzdPB_ z-vd)0P=BFb-iyJ3{l0(yu7PZg?p`WBIQ6O4Yd5`19*gTbV_a%?S&GKj*SAkCx?_8! z>b^RbE#opuunwc5to$yOWwV*dY_3P8@Olj^E9;2Y=J3XVQ)i;|*L=gqOlc{pp`oEG z4k#_cln*-!Q;ffMttEe6ll`?`_4_k@yxiPb!wfXXj#a{L zyZH%1UN$y-I1cL*TwDdSuTNp5cIV#Nc|h=|6WZg?IVZ_jDU4QDvpb4OFm&Y#8VH9S zmktUIeGKkotBb#W{W$N~eA#H`^kA8TuvzWHdIIXt&bo1l4TFTs_cN{0f~)K6?Vmm+ zmrV60Cv)7@zP~V(qQ1Mic>C^{jzg@a{7SnS5fjwX&M(P79R5EZt3ktm-y1Ki^(s{&fhALoj$$rWO*g@a@fw zI~}hoY`?y)y1&@drk$mxd2`^~Hx;YQ`->xK3eS3|Cc!MdmhorpQ>P)%#$^gF)gI~7 zLKYdFoSm&b;Iylai9yU$KS6~Gg%qqM-os_OeNjnC={@RIWTag0V%nz0>#CL94p-IJauXL@P?9x#_OxIfSWxt0HAAVEh z<>hr}A})AR6M1v<^YhoHRj-Z97RH-{?{p^qI=;Are(64)5wo*C%S1~{^eJOwW8bg7 zJEcV~(^@ zEOz?U;(Lfx%Xa)DyN^H7ig>=0r%r8eZJob(@e&lO))?W!+}yX7+J8-q?}6n*wIS}} zZ{{Qx0S&^0L8XiJ$#9YV{e2C@28EPBaUd28_#pAGzcm9cA^t@y9sjv_nm0K=KOauM zx~|QEmf~oT;+V+EGX^wp8FL|lish_}&E)$43c(SD<>lpuhK8X!#xuCENsE_jWF=?s zMynpJIRPJC;sM)`f~R<_eCAd{#c|!S5^PlMj6df9x%qur+|qd|d-e=PcUDfh(5|V* zt};it(##Ud#Z1bhCqCEH`=be$vF2%Otmp5ULy&^Q*Z3UBY<3S<-ItSXrOuJGEzaFt@%@t+@_85dhc?1afJcdeX^_|X{Hi|&-(d^Ne{gHBt!r{S6~n=O3d zLaf_zn9rUcxLnGARL+a0gA`?S@mj!U`2U5p0rNF4L!vioU%xsrwH{4LdN zk6-iis)-p1j{^f&?_S&L1c z+qZj$BNGx#H8jS?$7M60JbNbXIPSj__Q$?wWw_GS`0dTlU?N_B$@>c?1_qyG=$JDU zS97Mu#_BU6D=4?-U$2SF^x0ch#8USAmetiA7(3KNB_=NX{3%DFqo?OIcqfqap}#$= zz}5Rd(OLQAY>52a^3RXf@_Z^Is^qT*=+&i`+r&CG_96>i^@(2elK~$lgC(!@`%-s5 zNa|yBwfradAvP9Sy|cI0(+>Sk>`%GVs&cNehUKjOexTRk) zW~HSWhcJX_hVU0JniM|q*njx&;jv|y4av+X>AXd!g&eN_7*11Db4an6`3{Y5N5usp zvs#taoQmbo+(})k>hw5G2?+^s_uu_BUb}9VmM3%#Znwu}Tqq6vfwh7Yieq7txusfa z_gz+5>xRwd!mx?9^6Yu43liYrR`iuz6rC?c(;|~krEBrBl)%uY;PnZItd z74J}+;*%q&+6!T3B5~bW#X=RcbUdX}`Y=V$Q27$?tgPqq`g#Uk=4rqFOiC$<5qj}( z2!qQgt-ysje*M??Hk&r)2bEAmw%m_nVwl>~tHLvVP|Iv3>tfG4j+5j)EjnZXRZ=yQ>|0in&;Er! zyB}B_x-1hD6U9~6F~jo|tQLc$6g>mw^}M#;k&0h-sW0s<4La0iG(t>?*xFd%ZK>D( zDSh}*in6){Ef2oOq|RGfQ1Dq5H&3yYlvMHE7i()Sl{E%lKR!HTpVK9_c;xQxz- zi@z{7?@pB;j!Q|76T5T7tMdNh^@9JlL_YJmZ*TIMxm8})tK?Sk%BelOFMmi$MJ2sX zgU}~xNhuISRrykA*n;RY#pS}Bu zw|6DqhO3D8kb3(Q`s(|O$9eH0PgJnSP$WV8c`YvdoX=tld>*<=!?Xq`$Sq z_N$YV(^-cRRwky+t>qgz!<`z0<@rZ1Uf|A-wH+}}=qd?*@+A4TkD=lB!n%FF_KQcc zi0rp--;!zmzo8WWD@THoN33;7yN8Nr6FNo zYmb^u^2O1%(d)&inM5?~i)LL`Rnb zfCVr&t|f~~{$c|7qVX}+=)=`#11+CBJE@Nx`4DqN1F^yYHd~v}4-VMb@yI@Xph=wV0flLFT83!^0IN4_Tc~a5=RDdyA zITRtOsi_GTm@ThrV8ARSjziq7?Chjmro!*-Rd0<4o5oFl$@>b40RXsNrNKH_!P(h% zv;FubZ`N&i@|dq>T5};{=KdpqmP;X=H(J23Crh2(sR3%VYC55-s|x}o?t~ekEufVO zgV+P+8@AjFM<;r17@nI5O#5^E=YRTzHArW$dWJ#yh7)22!IOg%I8RKC}{B~!o9>T-uqv0o-CT4nX!4> zI8x*Jy{h&ijvEzJaLE{==QAp$g~653f|&I==abIOqz+WMyPjR8{pV0=@MG z0AS9C0n`f6*LAw*S58*qH6^7VGIV!SfA=X1aOBy&aed34A|4F!~>uG)g;~tN*#o$ZF6xnE6gA@@dx{~vB0@= zF`1il-y7K!fC!`j+I(5fxisRrs5xi<{cUsge?$FK8W<_}CG(aSya4VX;Ysa&`u;sO zZrm>JQQZw3zSr+C)ynd+1-oV=$s+8bix$hVzuoDxG4sV1Y+U9rn(6cB&!643X=ljk zKyrH1%Hc8r)5;h7QJ-&$-EL#P;B->8(75afarH_#kq0X`nV6Us%mQRDbAGH%<>%vj zTUM6k!+S(bOw8JHwARb+&2(>8Z%`bHm65SOH;jZaudvXp#3m{)Z}#~qWwj)k;S6P_ z_DOPpxr7ZJd~mi7pV;I9yUfFvbNAM!G>IsCQWO7?9!TrGAMdB8rXYfju%7Sg@89?r z&vP(f{ttNG#*u+|?YJg^_8%|~Q2oC$@L1j6yn|2hi^DFNUhsbtdj138|DylD zmlE&xf5rPxA`>p&zjg(+Si37kbh+XS9&1()9#l;IJQygk1<`@{n6Au}6iykL>ucEa z4_4BzWxNN!N2t>gI^S}JB5)KFD;{LY`BzI6!MiF1k%sd?+%h|Hk`RNmnpkVW<$7no7-z7gKB#A?AhzU(tvBZOaPcH4}}axPenzQlam7h-2Ko$AmD6J$`&Rs0NtxJ9hHsy7+jOk-f{PWVwjeU(vmrYA!CtNlENye8Al=^Wd<;!otBj zHLB+3$rU5l&^&mfiQm7q=?TYvME8rNOyWQKoP!lvYnqzQPfb~o7a}ga{QO+!zV!Ex zpSg^xKPi6pY-Lr|$L40kIojKI?)1{0FawqBG$#iK$NBT$-j7yp@9vTwK0N+~0}@Xx zR6hp?ht@@>z|e3$v`caMO|aKF9lj`))9<2_hIIAytIEr}%uLSnAjOAlN(S271<#%P z8WdWQoQ!PC!e`6+U2on@|H9^`qw~T<=;JRxG(KK~2j}hW-T!S?XqPtPl#dVCMVb53 zrP1Nxgkxwuk39(E=+8GlF@a(mvCQ>|{fFk>#S0fqYabqE!eNmDg3Ufi0^?Jz#yL8b z6%~dKriZb}Z4Lne0Un;K;#ofz7ujiO2(4HQ;&)X=Wo>N@*L&mLn>U)`QO6+nL_dE{ zr?0QCyvu1mXz_<4Dk|y+og50!qsFrz929irft!FCnJ@-aGIee3$gr@m@^`ZgSb01i z@xB7({y4C5dG_NjZlFuZAHW8?PV@6C-z-Eh$1bX=sYQREg+$QicP+!o(J`Xqn`cHw z2IIHQqr}%F_GyadPjYi>2iG6-*@bN3_F;6?EYAMM1jS|4n2#4qX_e}NE-L#3;UOmv zz7ogCSXo(lt)kVG_M1*=v49K_S%#E;0i5 zFf%hd`B}89jt0k3kt?kAtHk^dg+;g%4}Pz&udk~nl*zoZC?;*ng&9jD_Uzd+0PV4{ zu{_x8FRq zseSmpqvJ^7;S^*U3g_6wguehsqmrsBWLkF#GRzZH?sjmkFo!k`4(HEKdl_X(aE%IZ z5|K?i6zEk|Rx&a&rWy>e zWoS)LPd|lYs2>LpI*G9xETt;Y)Arj311X>6;4t4>`a!`Y6cQTR&q^C|jEpZT$cEHC ze)xJaoI*WatcV5r3mVkUOtl0LP&l%)d3=0);%{dw5UP_m=XxxtBYeD85%X zF@7Bj6-o=~2}~v&36OrES6~!y-}|=jcv36V&Qx01l+jo#nkQM#+R7??1^WSUQc}`5v4>mts%h;?(8pySmV5u%+1q>QA8y4F)+<5eIZsA0 zZ$Iuoq2-#Jp9j+0o-P8yCwx2soju7;N9XkO1C{@?t*h$l>esH(EpOr=?-$l=48R^L zswuUQ5wcQQ)Y`w=ckz>`f3$D>P}zCujVzVe@rj8hJ`8Z^E8kdhkzD@qZDEgH(P#}}P$sT+$zy+dS zc7A@f&q1v`n}NAGheWPwO^{N>{l!F=4`*`n^5n`+2=b4CQfX~%9qk-;`s7Ke?TN?) zBCyrJd-wG(R^xxv^BXg|Dpb7nbG#v+Bt?}sNbAsAT3WA7Dl9K_W`i208YiApRJ63W zy_%<2An80wiR-HE`lY6-Dxtr`HQN;OoqZ3>kzmll+x_P<^#PY+D0A5klxfP??c}2) zrvl27SvnQ+E!~bwd*ImEhL;<{^67gm%r^oVc*cT-YdJ6iX0;}pY5xJa+7eEHzh3b! zjMT8u(Vg*Fn*s$^>y&Jc&P@D+-3|MJEduHg>`76VGS{U@a4Oyh82hc(==H@>pHJbO zEa&ddjnrIZB0iE6ct|xm+_eW){PIsjA!4I%&}XdzT^50*7XCt$8V_CPeSdQL$+p*(kJ!vKZF;LdgQsx=p`(V=#GvO7)D z)YR14&qRvlP0+8d1}wf`DUkEk9}70@){j8;{gs$HjR_DEcfmsh`NpaiYtzVz%gVkE zJ*jUi8}*JK6My{ZQ2O;wdj_cl)l6!Q+5mCenp%!W5NK2CjC%gEoB?E3mO zW`BKoc^k+Bu+Qfa5h)In0H>j^Q+ao|GMGs?QQ9-w(_}XOphsC4aJ=w|6XRVea;2rE zaf=%v2J(-vMgIBpK^7k2to``&v*P07E}*pazeYypsQEPzHu0Ea=Ut|{eznFpzZd!k z({F>1Lo+v>+=*m&>R(~0s<(|h(fx#!l*y=y$2FqPf2Ow$6lBl|{X9KZJ1#j-wuN)b zS>1+shQ8XjJbmGXnPYE)NEpn?%`RwDYV&K2HPbuXhW3ep@@Xz2I zahFa>f0?D1*B@9%2}$S0Rk-!7TesHqZ7Hf(t%v_A|A$8sj=jO+%sH_@?BjvgX{$?1 zxG#-ChskL*M$lM6_o^wzK>zQ}*J~&c*rRfJ)fz7$F501HUC+ZqvV<|GU(e7mhh-zK zS1*PfT97;rl&5;8lC?E&d)TvQ3>pN9+mG;|?{6ic?dAmrAtloFcJd4jO7+dr~1#|^G%7fwF-j*YFcx%hu z)tQl8)!E+j`v1HPy9JGN-nmitpU>#I)!aQide%UwKD8&=HFM@NrEYyshBB9L)6(W< zOep%adSct|v-(u|;7SR48XA#19jrp%?RM6ti?PNE&I53P1_hE0Jwfm!Oljo>mR~Es zTG1C|S@mB0-6Wz9(v$}B`V4WwCb#}!Izn{Nz#yExezM(LCuB-nKJMk~ThDRL`-9mVy|ozgLpk&X62g3%EX%X9BKdoJ5} z1my|>HJ=cU>)fH}l^vaud%p{OKE-ctwC>UuZD>ZB`TTK}T!H-Kz1++c7u?-HH#cXs zSX@t3Gbw_~R*w!liCD8s072|}FaNytmy2T0v*R>0c%iNkQaN05;U&glT5l{CYhq&Z z=?oArZnalk1E&Q?Asw7jj%Jl|50|Y0OdJppF!UPex7?36|28#mcP3o?D3khyH$0U6 zlG`csxqud1*!s>Y5GyI+b%tvoSY4K z%+&1P&7v?xz8?(b5;!A2Lo^%f5GpIgoO0XV-MwDy_R*91@nY*ejD~4KbbdjBE=$*T zh~9;lm0-VQwry~jDQiUrxqg&Vu*)|xc?CXVqk@5$KBMYZ!Q?%?A-k?F!Pu^^f44N( z|2xq?ZZJPPJK6Q0R)vJW&3<)YN*;1H(rDUcLMQL*tImHPNi5S++3OLt9e0Xb8v4BY zUYeqI-l|CwHdLijc70#thOKqdE7>Af7K$4mbBk)Bp!Lp;xhK5D>>}Kwa(3W;Ffoka zG?M^TcDLo{3v9I7x7%x7Oa&Fq<+QxK{9OAuDf>f?d;%%;@$l!+S7>yo`xCtZ0MdL| zv;YiS??ui}I5y)V`Rms&pf9*rV7E%+uCIS>06z4NA8*$Glvl)hoSbDa^zPXo&2497 zBO1Ga&Qw%C$-buOFfHCN`ne55+Yq(0m!X{ZHHB^IN<8WvJ>- zRb3__yoA(HUSbcyj;Sx`_KyP0q6Krga3*rM&l%hIqcPAqCff435lt~-{~P9beqUEQ z*7Kjmtn%SGX0O62Z54?rjD%jS^-&_XL)DdgfU zT^UjC)QwNPrmq_fNO)545Nc_kfvvBAt@Tl8LvHl+@wk`gmi_$_=|L<@b#`=dxj9~% zO&heN+z|S~=>Wb zdqeJ>9%*+`_YbP~T5un0tOb=FFSwuoI<~H9zfe`Y_J?4VD8sgqA_t zVDhJqq?uuEo%`i~`!t(I%(=*j(7Ei(%@S%?=b`(X7Q}cTT`hfYs-rK4tf5MAIJPpD z8LilyhvX&#gRmWnO_cG9(*uRu7pT&1$rJ)zC}eg1UC)GxYk6 znzM_(a`!W9uaJ#M&vf*)H_@EuyN<~>t%{E3{OP2s2-8JWbh2R z3;U;F$L5{n%!-yLZ;-W3%nSP6533d{seNo;fEF^XY;v7kUGzDKk2Amqps^wnAbh9u zr7SF#;H#YX_qGp_9+|45dCW&BYo+pP_z)k)DH2&$MmZwA(qWf|exek4T65Lk$jWnL zZo0pKgDF6o2WjF$~6xU4`3A9+S;?YBKyI!!}il|6)D2s+*d_OZm(=ar}X!J z4qRv}j(-AFSr@B5hwHuH(wtr82>L0nMtt&;FFOe$qWUePFzHCVJnpn%n5ky zuwHZvxhcn)e$=jdo4H!-0X?VtX7-{;F^=P&4%!SWWKcwq5G^k}d9oRzb#`XPTZ}(F zh1d|AYCl2`!e&2P>f;u{6DJm4K zx$o~TZBNkWG#@}FgSh$n?Hf=3ZoFDMz*hk?wp3tqpzrj(NDVxELXmax z020g&2>(xa*WE!B1T`l?;88Uj4nI`=KulD08DJ13?I$O=iu}b5JXW~5xfQQmdAR;X zf4sKf=lJ-|GqL5f=h+I7g|3(DHsMn>0`&GhshLnBNe6#Dk(&Yc^%%Yv`Ex4m);9_V!` zgj9ndU#yv>o)mgtCzEh3D)9isARa8FAo3$eHda>~&*{zwS61{|q$9GZeqR%z&CSg~ z%mA%}!y=4{;dpW8Y*H|kde`t~LCKBgpceXNYK#FrNNu2cZOt1Eaz>uOkYS6i&CJXI zm@@Y*#b+qqatDqKgMj8yMcx8f3JOh}LU>ZrA}DG&7@Df&<>Na+!4dW4i4(E&S4UjT z%x9QEl=ZZ=RmUA*Fc^3Ve-c__j2h2+{kr4J7m!gHPa+Z$o(Zd_ zAfJ6l9GE9A6Aks;dq=4vhBJ3N?JNoA>M>B*W@h->B}JMql^oF1(>t=@QO^}pG_DpE zibkWa{tjho-6xzXqK;^_tfLT@-Ljfg@f|{p_u|ro5*=9Y!mwsVVr%q3XwBe$o)n@7UGZ**QgJb8EM#gCbpI z1a$U%h77HoFF%tZwjAwISa&1+RSKVj+u>-~U@$n7`-2CLDasbF;^J(Y+=!EirVDp` z0259@{2~1)>AXW|^m&YcK~epaXTcQDnl>? z8nWMtEF7wr6`Y*jl2wWYq6t%tF!iTRP1XJw>Au+t+{0E z>RJ*QNXqmY1P{CSkWiQQrPwcdOF{qA)6?_r#l}GDtp27T27VEu+vWc>CCH|1^(O5% zH7}(rMN%GICk)_VO!F515tux1p=jFT zG}_$!3p!#~NuVd{8W|rCHSijvxxH^oOA8Ruot+(^Zi$IYeG0_!co6pX*9(2#93v6j z2C|vNU9y9DdgImV}e@r1xxtDGwiQYJzH$$PVt1JJ7!~jyk_z$S_Yj$6Q zd~zvp3=GX5Kf=8K0gNu)dWP;m7fbz-?}k}^N{!v5nS(fI2=7dgoD~iQZNu0S{+&EW zOsl2~_?!o{1#>J5zwc*pA23T7v13GUcP83jZA1~LA-=W;__MLGF%gAJP>Sq6 z!_q-8NI_B%^Wm+X9d9T`Lv^!X3)#6o#YxzHd#-~4lgMCUqQAc#(1(QCx@NqY0SdH_ zo60>#I%fF4Uj<0X?%yesUn`(UMhHeQl!=TGhb|2$wiIA%&ksL<0hDmbPe)+RF}-_} z#TV?L$>N*1S5Z-M0L|_4uq750@yh^uf3(7*n}jhb@I?R$l{j(oKQO0>?-6{&=t(8} thaG0!70Va#5OAP}{QFf21kr0dKwVy35LrLPqzbPhO7dz~Udx(4`X3eV#)SX? literal 0 HcmV?d00001 diff --git a/design/images/tinkerbell_crd_refactor/workflow_state.puml b/design/images/tinkerbell_crd_refactor/workflow_state.puml new file mode 100644 index 0000000..3717367 --- /dev/null +++ b/design/images/tinkerbell_crd_refactor/workflow_state.puml @@ -0,0 +1,14 @@ +@startuml workflow_state_machine + +[*] --> WorkflowStatePending : Workflow reconciled +WorkflowStatePending --> WorkflowStateRunning : First action requested +WorkflowStateRunning --> WorkflowStateSucceeded +WorkflowStateRunning --> WorkflowStateFailed +WorkflowStateSucceeded --> [*] +WorkflowStateFailed --> [*] + +WorkflowStatePending : Workflow waiting to start +WorkflowStateSucceeded : All actions completed\nsuccessfully +WorkflowStateFailed : An action failed + +@enduml \ No newline at end of file diff --git a/design/images/tinkerbell_crd_refactor/workflow_state_machine.png b/design/images/tinkerbell_crd_refactor/workflow_state_machine.png new file mode 100644 index 0000000000000000000000000000000000000000..e85b4c3f0a83815b2508fe402ba3f9008d8233f0 GIT binary patch literal 16265 zcmd73cTkgExG$PSqZoP#pwdyg0s;z1@1RHvRXS1>klq9&(yNF9q6ku@3kWEp6r~rX zccg<-l-{JBC*QaCKKtJL$DKKIX3lZOQ9|uL3lU6UoZ~m9gemP`$asLCC!> z`b3J;yJ?|)ZMjA$R)>WY@2R5vh^7J+;4V&sWT$WA3h&&hdhc8Gonp;JNR_pl)3Fx!$^fJ8?&@l&nIk*vs#@9V$zaiylw!KXw0O^n&@++rSHtOrLuCE_l#| zW{REsq;8~9#%=JMWJZ%HO3CjdMGptRyL>%Xt4|_Cpkm(e{Q8h195=S!*xgk*`Tf-KH^D6j*s1{|Q-L6brhF#L_-wY%L6PyEgAG&R) zN;*7gA1{+$VU{SfEjAP?v#D7r6!9(9tCl8W)w?N4i)6(iSc5Sv!J5cm3LZR#9GaqF z_F!)#`I^)A`ubvTjz%n#tO9UiIDDVK=~_e>JLs~y{h-}G7;xBjSw zR3L6>Y~0F~=%Tp0 z`#RugFW_i{IpGeidiM9FzgtVg!^1d4o)vnm+V>g@%Va|YwdJ?|0{z3oLyc4kZmGoh z_(}HrdelPhuC5P$R}7L2ntgQ5wjX?(a^0(m$pUHB$`@B*Kghwsk^j1e?e=?zJpJPF zmD=N5=qKWD(W2twN57o{sO`6ZkB05LZZ8i{&!kHFj(l*6J-sW*!@|Rp@3S$lEoG!# zS7K3*I&lJaFYNqvxixAzUX2_i9M%ww{cbc)0HFZr&g!xkB@e06-ii_6~0qzNVxp`&X~o`$G5q3>7AxCRe5Y6SNfk?28G{o<9$ZCF;{R^Y33B=4Gt^J^G>4KG;x6>C`>p>67hk zZE5e{JC0`jcnDhn0L8~SPQD^h6G}&nt8li4t-@9k`=$TGsF#0OI zoSmJWX^y>q{kjgafsY2k1$At|NGlI^#bAtIqz8!NHqrlt&zz}<SnkY?_jo$T4nz$Hpcy;ppgSy{J!Pv9y_Ke{0aJ zG+M6iR`#o{D}rwjjrDl4k`l+!t5;8k4|5~DJomSM(_JVe}8b6jiSBOwU{dA_U7*m(Y4(lRUQQr$SVzPicz$!`Y*k; zpQ31lw2INc7?t$&X!SB&SfxV4!eFP2X`Gh^E}i_miK9`n8-923T-|Q?8zht6pFe-Z z#KdyD$j)9lN$R_Hr9%_B>kVPm*FjmHVOo@dNO(0^Xvof#Wl-b!+bHN1=_;kVod-Ic z_ES($eRrmE9QpdeR2Xv!m2Yz^+hnovN7vQKFJ=$-8XKNHrAbSQ5QHo^H846dVi{}4 znD(f*&`^j6pA?0{Cnw*AFlpE?Ie+aw%dA4C3_h8@!f?&zZNTx7_m!xv!@pa;hg*Z$ zcY2hAPygBS{5_H)9l*q_B~qss+KO|Pa+@a^*xlQUh<*F^E#q}hVFxrllf10#7s!!P zNBHn)`Q43fW#&VzaVK)I@M`xzV`l{&rd5l~YL6bBW};}1e9=PP{v1ore)TG?+!wX$ z*FWL$2)MO4)Xk~I{sQU4O&xUx)ry~my$dA!iy2q7w6$GhKk0QeY}U(QRsU|4dZwXN zD!B0mY=pJ|Z6X&w4V?nID<%y^^j|MQv-)4_90p95x0tLjUHgg9hV7wPT;Sy7C%-ePZgonUDCnff=iy;EXUJOV6h={U%$%HQetIkdYD_*S-Ekw=_>&HmQncl6=_+P0t<* zWV&r<>eYa$$w>-k=1xm^cZH^&-u(15+dX8OS?xp1zncHNk&R96GhM?(Nr?b0w9G={ zlv}p8wx`h&n_cn-Hjgxs4T}cv?wp~apr>!iNCTLlZD2rzp8ff=j4KymnVprDRYH-A z;LT1;W8vViu@+mQSUNH2Km5)*`KqHh>@(KfgvuUW>cW0pJ zwul~M8tQ2OK>5I|#_xrfnAT6o=<`<{(op567QOwBsHw$VXI13nP}sD*odAd9XKJPQ ze~5o0Ns~h#lzOjOj`w1(*bnfP2OE+F0#>rOp%<~&mx17ewH9`>hg`~%6;n^X`g^$S zCv<2N6R~S|duC&je~YiHtAmDW_z~qis-Uc_gQV?jOW>_=H)=F=w(ShEv+<8^IRyp0HBV{qMJ1USH}9jppRgt2 zCMzQq?LU5)qy}A0gEVu)$>iKh8Go>JNm*Grcx-oVrgZ(&N{#$b)FR^7=m&n3HfB;O z<{YD_Gotx;cb<~t zmw6m5L<>GH4+wU}HbMx~V z@6EW;<{$m{x0#rjrVDa&xoU{vmhn|uh@`8IM)jej6Mh@t0AD~Uy!qhsd1CX`A*zxnxf0RU@`1iq2ot5w|DJv_RdLYqX zU0uC)L78FPYr;QVwe(>_IOUK6Q=P1=?DPy2e29f`kxNG9LzOO8(4Q}&TRv#FPOZqczLM~H z_wevMz*o*Efq0nQ70v zj1JF}syW=gy>-VU(65H2PxIj5fTp&}Yx7F+v%>t(tk^VJbjNFHW{vs4K-3R2t4Wlg z?^vhw@mRhtud#>EgRMQch7V1azO7F$wj4lF;KAgk?D_QBo_#4`A@^F!|7L3z=F`>0 zkK7*@vb8$e;?Kn2l#u1AmOSzKWs?!WpRi-$TEf!%%R+u0P&ZBXR<8r@ z{@s&(1JY%GXNCXu2aS%mLCGg3BGcDrOBrcsf}7B(2hhUrK^eH>?(8gss=m{-u1sot z?u8Rzt=tm7&2HuC8Nf0-hGjObosIDgl$4a2%CYyMvO}iO@)@Lg(SD7vW0(>s*>mfZ z%qbYi3nmfL=QgOm+6M)h*?SVXOv9nup?27pMSQ1{6>Hv)sr1`(%~XnMC_Hb~{5VC* zzbZ3xrtZ;+H9=9)p{6FKs+cDZBjtAiQ#5>wiIi7Ui`*2b1su=pw<lx32m;HyMqWYc;x2e4e4yQx0&XC0($zccV1Z}w_b4$iTACMOLw zH~)l9`%&%tAuQ}P6@cJhx~fl~JxfVRx$dz@sHA`$m-0P$SnE4;`o+#8Z*U?^K_0c#Fp{h-ph)Twh$fmzP!;XJ?Klf+RFKQQj0xMbDVLorP9~0-|l0VI#`uwAg86cu(!7-WYZzw7i+F&xh#3*O3&cnPF_yx)6N5n zD%2kr9#)_EMKrJN+H|8zgtq_A53f?Q#u?$cKFBZ`3mXd@pG&j1k57f;=yp@g`4WUw zq|u+@GTTNSTz)69v9Ym(qoXbj-i9;(rMG7apkFz3?$6k&!`aqnT8;jH7jo7v(ABTA zZD4uz=*Ke`BW|dvX)1;#cdecwB`s1yCrKqdk#O6h9%L66KX(Z&c;l%B;PjDdF1!%2 zy&f{-C?ffk1-t3F%%Z4RPft%aQHI*M}HMW?UAQ$&K zUy2lXEDe}^@>G;$W@cn$?2cFb>r}JXw0c!nBl^8?;x(snKHZr)A+mNX%QxBFfSF#R8<>c5x5m1b4E4#ARN2AfDaQ{b@BH9i)mNGqjImcAL5xMm8iH-T3s_2$K6Uj4Ue+LGpV zh&7jq8h;gPdiq@NWWA)_k3UoGab&lA#wuNQA-XCaoQ8zDv`9AH+`gEiro!)*K>eII z4O(#v#a))Ap&^6D!kc0ujil-do3!bua+w6%k5%pqRDYNjJa82{y?ey#8yk(iIW_Li^NMgKom&hGiv3C#d8svuFr40pe?7Ql&hJQLSC!@DLE=as07 zC2Xfr)m|R{IDnssxa6`74q)cBd)KV3t#NppC6pcIYBfUe0xvJ|PSZkfPKDQ~4ie*Z z`e*0sTKmDv^+%|NkxIZ4mX^~QicxHAY-%@di1^g&AeBbZ&@e1}lDxvV9xc^aHe!VL zg@l9xjt{*^Cl+xC!N`~xQexQ+0>k+4jN|`BLjI4+SKTxJrA0}i3k}Pf%$nlZl}lV< zuL0S5_y*5AlT>sjUb+pGYX^F&4l*l|#@yC+r%RsP=df|S+PBnosKnyYqepLkCYpYi zWO{1Kksx_$BkSEod61$s>u&H@yT^WnKh?X_QS(nCB5*gNUp$SNR?Tw&t*}h9F8B=f zIH)mqeY!Igb6Frz`J|*oZ)S zB4|-Pg+Q)$SEnou1*vt^-h8drND)uKuv8S`0sUmfKLY_oPh3o_dV9pa#}%-NnfDZW zY#Q*K+}QV5Vr6M*h&9idCejNs#vq(JL2?ml8FY@(Zdm4-nHe`%SBhH%{pIDmckhrV zPL(^3(iR;8fO`ln8{}G%4h8X3eoNESfvYMvZz>F#TY!{Wd$^sf(wnQr&T#zb#A(0_ z!tTE`>xB}Sk3m(kz%<-eQfgW%uDy&wu7w`N@rjA;0XWx8pb1V9_aNvdEri++dPP0T z(=q7`*}Jb#wZ=59S3sq_h%R>+ey1;H_mG#z9EyiF>3hH)pnvry6%^M{Em0oqRt6kDthOR1B^7yt;Erj*DEI8} zYsyo0?}7{kZ*07Illw*6pA-zsd@e$Zt}Z6(%~l|qIP3h6(a}+7eh_wEY=#q`IYWs7 zLF$5DhX4Isgbzp0LycFWk2VDb1uYLBGhSb%e1QYvBKFy1NzW&+u#m4%K}XJygq-|2 zFAEo!N+HBe)o0qY=Mmf&b%v`!XI9LZ{;8m5-u3jnA!dooo9NbUI*fj;{V@)?s2|;&f}oH;5KZ zH$uCqu9nmT#eQ1kWNN>1ER!Nx%kMDehjkBb-@ZK?`*-hq%5};*Z3-!X2=ZZMitj#* zKOp7}`R{A}0tMAz2L#~rmnSM+W|O@Nzd%@HKGI9JHR76mRDL2Bw7kZPw- zp43EU>0vd|t2cWW+(SLgB~HLC*(cRhR?=UtL**mZN!RN5xVZ%dsqv?_fiU1M<;2!= z@$ft?28{H0w@9m47j*ia0@%-M5 zG2ur8k^ik4QusyCeRDDp)lO%yGUZAj9l5%`%Ru$?^Z++8Q~>MailU+-k?&n5VpXJz zi;KZ!KE6AY;Lyx$F7%%6PK}Shz|A5ef+YrgV@)uD_Oo(wM&nNqPx4RKLS`B~;h^Dw ztB7hy?cD)3UgfbA+Cmqg98Le_AC2hAlP6u7%H+wlz$mH9rm4W3LHER`2G;oRN9NTK z6rvDFFVHv|q>n;FLP~*-S(^NEuYzEr$NU0@J;jg&qS=Tolx$6WZLM@vAckc?#lnJ> z=E>{VodgLf0a)Z$Irxv^1^}12*;yzXRzP(@d|pYC1h`=g+!xx)W#h`ac4hRFe7NnD zqT3%b8kx?Y2P?}kMRDzi*95d7d(e`=$ePX0LaAhn`TyMnKdg3d{(MTvv-o(>sG6A> za}nBSGBPq&X9w?wK$K>R*9FLfN2MAZ5<;NcIhWMb)Xb3sXm}rTZEY=#oO$3bWgmLz z^dk(*J>x2OZed|ujtSf`0W6!^@!+|r&z|+!TGUN>a|VN%4tawLM?sY$BPZ97ea$K# zD$JDwd;*x#_7SaP*ZV#E=f|h#hm?7U41{Io*Vl;Cr04o7?Ddl@A-qBT z3L^#ztt*cqA|ZKo&B;`KlMesU`y;%&69kgm76&r_ibc@k@&Qp97#L);tpe88-Ob0e z;_6JzdM2UsGlhX>s za4kx1$)GLC1(B~~bdAKv7ynM;@bLD;Plc!Gu=b+u{fWVNai-x9a^{Xo#c+Nxw zHD`nVty{P3BBQKN2K?qiv_r$(SzV;jh}o;kowbAUXbCToiX4DTgf1k_|ewJ`dFb{bUg* z6EZtLKVWfu3~Ta*420n<(8jc{VRHS!$%Ax1M@97?GR^SF$PrNFf0P(SBxsFWtEhGBPre&oHC(IqL4+yQ3eRel~*dmq%+*Y8hf< zbP{?eDKW7;I!r$i!;)Y}35vy;1lt?2$E4@4g3!GhsEy3xfjy8r34#f@GXU(gBg#R- z9K?y(^=iKJb@9_vr=#oVD>6|kj>2a_CxtqyoqWCVPl=AiZ^I*kV+{He&`_Z~I^g5pl=X_ETkL-_-uq@@?S2&p>*=1h0># zG5(pKPki;t_(ND+V&W-c8ald-NJ3ME;&|rF8No32GsMI*1z_6u@z_1`_3?SpSs)6w zDpZEGDtO;j6%7q94-b(7vmYMfDW5+1-nnz3m<%g(kG8yu^4MmV-wW{m_wV1qxpg(XBQV$b@e%2SfDLUZSBC_4{2#_EiHKTkQs~+ z^y^fB9P+;M?XIXftb?f(4}OdnY!Zl((gyahj~!XZY%s2n5F8u~(8cgqr-L6iq?9k) zwQC>lA5)f8nPO;YY1i+;{<@CU`1?XuLS1>cNewsV=H>NM8k%qxReVCi-tI1+;X8%% zIn*>X!d`!jP4&Q;Qd3b0Pi&g~8WDX~h(lcbB_VpSL2b2whJey1-ooSrJ)0)1DozGk zrQF49*YtgJnvUdPh42uAnO_+|i|VJ24iBH)g%<@Qi;GtXHiG}-@$qp#5ui(Gzz1Lf zh)DC*-pY}`5mtOE53w$tm7N_!FZfl>X0olb)63a;lJFe|&!?v?;jun{mRJ&-1H318 z{kmHZ;TK|@olsvz4lrHxHxl{QwzlT{?tt(`&A{L=Z}q63!qP8Y`~_(oAY(Z{zt3|= z?32|&)Yrn!#1TRYfJUaIL-#R}_X-%i85F|9*hpxi;re(#-XrC`BqFP(Xu%B&tvrPG zw{L_!AS>e)1gUPxj}>4PH|&L!l$77u%d8KS+@!AX)EO;SgTr~CI#yymJ3ak%uO$}| zqhnz3YkAp0w;F$wQ%uYUHsOGijYp;o%=oeCWu{^|bMv&sCbe`_%W=@?NPByfqylanHy4>EtrbVzEiPcbG(-6WM&TUADXtnO1rpp z4|y-b(mhE@`Mtc{v9S4vvOK+T?>H|HaYM1=Rr;f|$Ztr&8jAKu7ykXR#ij7ZEWHptT{SffU?~A*kQf{t92$;4zopI{b?~oV`R;P8|Sz9;&}*PO$W0Rcc6_= zPIdttD=#nq@yTm{Z*MrBnu_Y{Zn5VTAt88Hn&Y2=jwgB2$h^1u^*;d}2g9<#4`6s{ zG@5>@oxk?ZRNpI4Ejoc^{)Q1GR&zA%{qbr&`DhE|uh@f4t{ZO%$m+_gSO?(4d}n#m z^uAdA1OkBlG65|2((U&pmyNE62e8TM+<0rXkYgx^?xTW-F{yGdt*9_5F-Hc~CUz%h zNk9r62)92?N8KyN;(hP0&$cp<<-V0U$eg>#VI374+dMZ}8HAJhtz}?fHz$O8dBTo3 zJiVD<#X#rT@TjZiN&BCsW;=4w{=ZDk==WyG$=25|UT`yqAitGfypbqzxY-L-`P@|p zdaIg9jEag%og~fq^T`Pb+3bm}A;ff4)YNofytvFL71boyonWr;U&9Qvw-f&vg=b2W z4SKM>93cXaFm?Wm8t){)2N1Bp?jm9RhKQYcnJVRPRPezATqUOc@I9~@K|mND9VO;A zf8lhJHwV#o0JA~-rk`6%x|u+^9tX23G$0@1@U?`GYq4o{*2A-!hzTM+Ii&Xy`w_oc zO^01K4|D}kXM)~4xkzk7C0qRUi;D+cX7kN*C^;X|huu3ElmN|LRYj%iAcl#V8Aw_H z436M1^xGWVh!5z|W4*6H9ttg3dV!pDb!l!>3hN@F(OoSVrTHEAj8kKNcJ=_wHX9G~ zT!RcPliuV$F7U`cq&^~n92+t$utfhWPPY5~>4t5USgkv;)pmEJd?8&vtmEgdD}6ebC~+{XE+bO%(KuH(6N(DG%mgpgU)y^68BK z{QC8)3iEJVJgWFGQ5{b}R~?BB4hR6QT(y&$Hr{(GM%YwHsVbO__QMO+M-QDYQ(@0G?*1 zqYFzuuY1Od7pewjx&1)l84?l)82N$em6k6<5AFogKir#p?)7JEVR12%B-{J#SB`o4 zOT75%TlYw}#+eh-@dWt`ya|O_f~>~D0iwJg0N!vJpmshSJiKPUzeTv^y}Y~5QIsv==6<*ktuZP#$dKENRg8H8-2$4#S)_<48&gR;Dy03wMw zCqhLCghCzUeAm*QmM5&FN6&$Z=4J&uYPWy)X)Ks?#(6-GN71P7bHk18;A_)DppcjfoL??+TO;&>|F? zYoLpHEDs4TxHvi-TE$8Zfcyn6NN`vf4BfO%{MNOg=EdQ~HpOL0W@O)~hdwCo{f}va zDin=xfB_!m+@(EO2h{OflAt`5=fC7(zze2@E?*`ib`IWMoNkQjl-lQ8ew+YmIgbVf!;&uiEeeqO2BhnmkVS&bcJ1}^}T!(bS6Y=FvF9Y=6gRk}kB7_2#&v>1D6 zcO=edEo-Q(J_*H_%^7aFLf&pil*R=J{Z^_(o$EYG#m4pI|_bLs-Hni`?r8XTO^1(&Wl zj$~yZZY-Z641P}5VoEA1BzNh2e0;PUnZ#GOw)_|Ka?>4zxsV|Rt{6jD$4XOW(i$?R9L)Rku0fD{L<{M{jcyAQ?<$r!G9}h#DuZi}bn?AS; zCJy+TU)Mok%s&$Entl+Nl#d6yp!kAbXl!ik7MD4&mB?BmiJdXGm^Oe_jMwhJ>Kp!t z*(qej@!vMqpcp_A*(0%_k;K*wB}|C|r|OT=DCFed<09rRvVmh-XB)?V;;Vy4s8W)@_qQk#m~z zR((MIWYBaA;J^?aLC_P-R+w~H*Ydilg)dz*AhF$*HOU0>05Jy}yhE^(_a_8zKVd>G z_GA-NZLUm63kqtrsa=SQiTOQP%uJ^JhZ}dBXxYyCv z&N2Zq$1t__A0sitOwtdH{X)J`sz9w%OdX1wnmU{8uVQV+MVem5nd*%ig}%+Y3w2^V zLy>roMF&z+`qlctZ6|^?tHQBR{Pl}1Nqf3db1s2-S46qbwP6+L0+H=RCnzuJ`qHs5 z$11vnGS_2D;%x~nVaDh30GJayL^m!ZK4rBeXAn)>UwSPGV*xxh4`9xg@^#axNIi*Y z%ua}=w9E9#MCrX*>ScSFkjRi1?QvfkP*=0DvSKGqipoDJhg->tCY8$6ClitUCxmiM zvH;~pQqo*$`{EdLuSl5n6*oqc;h#IeqQjern1XurRX9T4`VHFUu*`i8A#QAj8S@$T z)F!AaZ>OBQBf~8I)^xSWwEg;Jx7OX>e&gkWOtG+LjI#U;4FC4M5ZH}273S2HsZg58yNx&L7zLoIykC*X>kZxrBv@JUC4q<6=<3Z<#9 z+y>)wDQAK@eV%bn_(OCO0;;mI1X;mXfIxw^S&p;JIy`)=k3^GV(> zRgW1>(oxYd%^9Ngv6WldoaqvsFtcMm3YintC+=%Q{`thsB4A-{R!u<>lfc`m(UH;*=Cr`o(s$jSxL~ zZfN=?4{ho`yJl*^4~+M(!i35dc`Yn<^4BjStA)7`HKOl2H;FAAZeqonnyj>5O>tho zeqQ55A*E1w+eXJf&af~a@BBHlihBv=#6DNj5eg}qGJqr-E_X%JP8EesTO?KTrz7qT zxShNQ;g)qHDoKGDb>>MYcxhSsPv4OI6G-Ap>OXj7vIiqfuWXg&jy~o@jY)_SS9^8Z zz}+w&CqHTP=8F_MNV{df%y_-1^m!N7}LaWQKq!a^QF?v`+( zx4)Z25ooqDI%#}KVJ(a)@5nfeQ3FR~4qpR2{}0w&w5(*I`c^bLRm@#N3x_xSrkOSO zB1C;T;!CH+1zMbaCMD+zN_{s>j0V1$T<6 ztsd!r_|J={4`pU%isT}weJwSSHW_%(Zb3Cw-7mrwQ~P+4OD>koL@*J|`C4Lu*JmRe zPOPWWn72JS|Hi?x^4-_R6g<4xe?D1&DDhsKekmFPPp@5ZLP)7%=cw}Gp*W4X_obV< z_~|F-?KD+msB^1?(UoI78dCxOou$Wv$ORZ`l3?plJSV zbJLwCZ63Idike!6r~Na)v|ki-^$FJkJxj4sDPKxegdt^(Qj|ph+!9fGrE+~jsH7n7 zG3YlN^WWsp(d*w69~mAlrPb^OX*4bJdrKiBP|iSSozE0+cw}5!NUgrS8Df^V&YreA z8|jNP&f5`ku*N#WOw(O~*I>FL)e9#l&0*B#Qro(m zq*0RREzBI2W@>987LV0m3RBinfvnt#aiWuW!p_Eal*`_AGetV!2$&wtinum6I3Z7x zk}BbBgmD*;Eex@lAriF3brpfDvj8wOV0Gb01QZVU^)8U5gep|KJG3}!0Jhx^C{=d7kaW+?9RoHMq+upmODE-4nQ1N z=kv5iA8e00m4q7;7B=h68^DcL)}Uy@2xGS{y|5nyt+fOOVL|&=dYDOn-j)5@wZWn4Y}jv?Y)fULM#+~~ z*2Dg4aZ}OKhH|umaZ#v=%oWTAkz@CA^EaqmN%8Rw_~KyHpRtcY^Dhz_@VRCxl7dA= z3*n(pL!Df3LO;gFXs$O=ilFfzxbu>gWQ`*xGT+!E6VA(c(C^U*W27nFSq-cB$LIOk(Ks&m7y07{Q1R8u`>CmjI=Ac-Q3&^X++eGjLt=h z3kf|_ltq`^yXPVE4$O&MbP0@YyPb4Ig8W8$Ol0-}gfo?^moGntcO~(Ii~@4v!HfKn zM+VK&XX{}c2LCZ=vN@&7bxw^|aVs(Z`Sa(mUo-zuLmMC(8yne`qI2s5z7O6ZCdQZ_ zz-Y6)g++QkiNPBa?>YpMYDR+zAeLJFT}Aosg1yN6h9YMhDZJW{mXpjA7`S%X-S*a zKyvSwJ-99^YVhl}`(u<6`pG?(D_5u-??&xkGC8Pf9i|3>n3OO>^sb2xr29r>8Lqh!LZXGI!q>oiqv{>v~KJ?ZRikYaSfL?J{)<-6A>)JZB^g=cyi$gZP!9Xqv8-`Lr zKY1w^&2O2?{u`{8>Ag;FD!Bd(tw6mBwFyE@iwgXPpy+6lvV5ng%Xm;nrn1vfDwkN5 z&bGC;7i_S{aj4tC_JM@~L73}1_+8e8d^;fk z!at_kYl zMO~&9Ji)&T|5!#i{uFIOGy^Ia&uq|0O&0#$yDLC+2s=a3&LSW{q2WOJmS_e%T76dP zlb^E?g63zG2{&xbvciWK_yg9u^(Y*xgoA+?v;he4!I+XNyap6aHMO>H-x8Sm!VN%T zR^b0y2}k4X7;*2hU7!y-@XlXPBndnyeLg#TXE`ng1y4zN_Uu`L$HIM@d=-z!!<6e_ zcehsI3s5GlZzD5kyI^Jq$e-LrI19waYi2e9%SS5;M+oD<&EnR1Gei#~Www94Q?s+P z;Y41^P}jhKUK^;@Oct@Qg$9V+w_zb6TRUMff-9Ntx${pA@U~PJT5r?q2L^%DsLYVXCLm#oh zGqtM+mAEHK%fA3y)#?VJ0`0Q3a$DxH=r*B4UNp(hcwSq$9RTVy3fxq{E?j OgsS2#g|~9%f&T- Date: Mon, 27 Feb 2023 08:28:29 -0600 Subject: [PATCH 05/14] Change action map to list so we don't loose action order Signed-off-by: Chris Doherty --- design/20230222_tinkerbell_crd_refactor.md | 122 ++++++++++----------- 1 file changed, 60 insertions(+), 62 deletions(-) diff --git a/design/20230222_tinkerbell_crd_refactor.md b/design/20230222_tinkerbell_crd_refactor.md index aa0e631..727b74f 100644 --- a/design/20230222_tinkerbell_crd_refactor.md +++ b/design/20230222_tinkerbell_crd_refactor.md @@ -91,28 +91,28 @@ type Hardware struct { type HardwareSpec struct { // NetworkInterfaces defines the desired DHCP and netboot configuration for a network interface. - //+kubebuilder:validation:MinItems=1 + // +kubebuilder:validation:MinItems=1 NetworkInterfaces NetworkInterfaces `json:"networkInterfaces,omitempty"` // IPXE provides iPXE script override fields. This is useful for debugging or netboot // customization. - //+optional. + // +optional. IPXE IPXE `json:"ipxe,omitempty"` // OSIE describes the Operating System Installation Environment to be netbooted. OSIE OSIE `json:"osie,omitempty"` // Instance describes instance specific data that is generally unused by Tinkerbell core. - //+optional + // +optional Instance Instance `json:"instance,omitempty"` // StorageDevices is a list of storage devices that will be available in the OSIE. - //+optional. + // +optional. StorageDevices []StorageDevice `json:"storageDevices,omitempty"` // BMCRef references a Rufio Machine object. It exists in the current API and will not be changed // with this proposal. - //+optional. + // +optional. BMCRef LocalObjectReference `json:"bmcRef,omitempty"` } @@ -122,48 +122,48 @@ type NetworkInterface struct { DHCP DHCP `json:"dhcp,omitempty"` // DisableDHCP disables DHCP for this interface. Implies DisableNetboot. - //+kubebuilder:default=false + // +kubebuilder:default=false DisableDHCP bool `json:"disableDhcp,omitempty"` // DisableNetboot disables netbooting for this interface. The interface will still receive // network information speified on by DHCP. - //+kubebuilder:default=false + // +kubebuilder:default=false DisableNetboot bool `json:"disableNetboot,omitempty"` } // DHCP describes basic network configuration to be served in DHCP offers. type DHCP struct { // IP is an IPv4 address. - //+kubebuilder:validation:Pattern="(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)(\.(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)){3}" + // +kubebuilder:validation:Pattern="(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)(\.(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)){3}" IP string `json:"ip,omitempty"` // Netmask is an IPv4 netmask. - //+kubebuilder+validation:Pattern="^(255)\.(0|128|192|224|240|248|252|254|255)\.(0|128|192|224|240|248|252|254|255)\.(0|128|192|224|240|248|252|254|255)" + // +kubebuilder+validation:Pattern="^(255)\.(0|128|192|224|240|248|252|254|255)\.(0|128|192|224|240|248|252|254|255)\.(0|128|192|224|240|248|252|254|255)" Netmask string `json:"netmask,omitempty"` // Gateway is the default gateway address. - //+kubebuilder:validation:Pattern="(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)(\.(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)){3}" - //+optional + // +kubebuilder:validation:Pattern="(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)(\.(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)){3}" + // +optional Gateway string `json:"gateway,omitempty"` - //+kubebuilder:validation:Pattern="^(([a-zA-Z0-9]|[a-zA-Z0-9][a-zA-Z0-9\-]*[a-zA-Z0-9])\.)*([A-Za-z0-9]|[A-Za-z0-9]"[A-Za-z0-9\-]*[A-Za-z0-9])$" - //+optional + // +kubebuilder:validation:Pattern="^(([a-zA-Z0-9]|[a-zA-Z0-9][a-zA-Z0-9\-]*[a-zA-Z0-9])\.)*([A-Za-z0-9]|[A-Za-z0-9]"[A-Za-z0-9\-]*[A-Za-z0-9])$" + // +optional Hostname string `json:"hostname,omitempty"` // VLANID is a VLAN ID between 0 and 4096. - //+kubebuilder:validation:Pattern="^(([0-9][0-9]{0,2}|[1-3][0-9][0-9][0-9]|40([0-8][0-9]|9[0-6]))(,[1-9][0-9]{0,2}|[1-3][0-9][0-9][0-9]|40([0-8][0-9]|9[0-6]))*)$" - //+optional + // +kubebuilder:validation:Pattern="^(([0-9][0-9]{0,2}|[1-3][0-9][0-9][0-9]|40([0-8][0-9]|9[0-6]))(,[1-9][0-9]{0,2}|[1-3][0-9][0-9][0-9]|40([0-8][0-9]|9[0-6]))*)$" + // +optional VLANID string `json:"vlanId,omitempty"` - //+optional + // +optional Nameservers []Nameserver `json:"nameservers,omitempty"` - //+optional + // +optional Timeservers []Timeserver `json:"timeservers,omitempty"` - // 24h default - //+kubebuilder:default=86400 - //+kubebuilder:validation:Minimum=0 + // 24h default. + // +kubebuilder:default=86400 + // +kubebuilder:validation:Minimum=0 LeaseTime int32 `json:"leaseTime"` } @@ -175,18 +175,18 @@ type Netboot struct { // KernelParams passed to the kernel when launching the OSIE. Parameters are joined with a // space. - //+optional + // +optional KernelParams []string `json:"kernelParams,omitempty"` } // IPXE describes overrides for IPXE scripts. At least 1 option must be specified. type IPXE struct { // Content is an inline iPXE script. - //+optional + // +optional Content *string `json:"inline,omitempty"` // URL is a URL to a hosted iPXE script. - //+optional + // +optional URL *string `json:"url,omitempty"` } @@ -195,11 +195,11 @@ type IPXE struct { // service such as Tinkerbell's Hegel. The core Tinkerbell stack does not leverage this data. type Instance struct { // Userdata is data with a structure understood by the producer and consumer of the data. - //+optional + // +optional Userdata string `json:"userdata,omitempty"` // Vendordata is data with a structure understood by the producer and consumer of the data. - //+optional + // +optional Vendordata string `json:"vendordata,omitempty"` } @@ -207,20 +207,20 @@ type Instance struct { type NetworkInterfaces map[MAC]NetworkInterface // MAC is a Media Access Control address. -//+kubebuilder:validation:Pattern="^([0-9A-Fa-f]{2}[:-]){5}([0-9A-Fa-f]{2})$" +// +kubebuilder:validation:Pattern="^([0-9A-Fa-f]{2}[:-]){5}([0-9A-Fa-f]{2})$" type MAC string // Nameserver is an IP or hostname. -//+kubebuilder:validation:Pattern="^(([a-zA-Z0-9]|[a-zA-Z0-9][a-zA-Z0-9\-]*[a-zA-Z0-9])\.)*([A-Za-z0-9]|[A-Za-z0-9][A-Za-z0-9\-]*[A-Za-z0-9])$|^(([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])\.){3}([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])$" +// +kubebuilder:validation:Pattern="^(([a-zA-Z0-9]|[a-zA-Z0-9][a-zA-Z0-9\-]*[a-zA-Z0-9])\.)*([A-Za-z0-9]|[A-Za-z0-9][A-Za-z0-9\-]*[A-Za-z0-9])$|^(([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])\.){3}([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])$" type Nameserver string // Timeserver is an IP or hostname. -//+kubebuilder:validation:Pattern="^(([a-zA-Z0-9]|[a-zA-Z0-9][a-zA-Z0-9\-]*[a-zA-Z0-9])\.)*([A-Za-z0-9]|[A-Za-z0-9][A-Za-z0-9\-]*[A-Za-z0-9])$|^(([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])\.){3}([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])$" +// +kubebuilder:validation:Pattern="^(([a-zA-Z0-9]|[a-zA-Z0-9][a-zA-Z0-9\-]*[a-zA-Z0-9])\.)*([A-Za-z0-9]|[A-Za-z0-9][A-Za-z0-9\-]*[A-Za-z0-9])$|^(([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])\.){3}([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])$" type Timeserver string // StorageDevice describes a storage device path that will be present in the OSIE. // StorageDevices must be valid linux paths. -//+kubebuilder:validation:Pattern="^(/[^/ ]*)+/?$" +// +kubebuilder:validation:Pattern="^(/[^/ ]*)+/?$" type StorageDevice string ``` @@ -251,8 +251,9 @@ type OSIESpec struct { ```go // Template defines a set of actions to be run on a target machine. The template is rendered -// prior to execution where it is exposed to Hardware and user defined data. All fields within -// TemplateSpec may contain template values. See https://pkg.go.dev/text/template for more details. +// prior to execution where it is exposed to Hardware and user defined data. Most fields within the +// TemplateSpec may contain templates values excluding .TemplateSpec.Actions[].Name. +// See https://pkg.go.dev/text/template for more details. type Template struct { metav1.TypeMeta `json:",inline"` metav1.ObjectMeta `json:"metadata,omitempty"` @@ -264,34 +265,34 @@ type TemplateSpec struct { // Actions defines the set of actions to be run on a target machine. Actions are run sequentially // in the order they are specified. At least 1 action must be specified. Names of actions // must be unique within a Template. - //kubebuilder:validation:MinItems=1 + // +kubebuilder:validation:MinItems=1 Actions []Action `json:"actions,omitempty"` // Volumes to be mounted on all actions. If an action specifies the same volume it will take // precedence. - //+optional + // +optional Volumes []Volume `json:"volumes,omitempty"` // Env defines environment variables to be available in all actions. If an action specifies // the same environment variable it will take precedence. - //+optional + // +optional Env map[string]string `json:"env,omitempty"` } // Action defines an individual action to be run on a target machine. type Action struct { - // Name is a unique name for the action. + // Name is a unique name for the action. It cannot be a templated value. Name string `json:"name,omitempty"` // Image is an OCI image. Image string `json:"image,omitempty"` // Cmd defines the command to use when launching the image. - //+optional + // +optional Cmd string `json:"cmd,omitempty"` // Args are a set of arguments to be passed to the container on launch. - //+optional + // +optional Args []string `json:"args,omitempty"` // Env defines environment variables used when launching the container. @@ -299,13 +300,13 @@ type Action struct { Env map[string]string `json:"env,omitempty"` // Volumes defines the volumes to mount into the container. - //+optional + // +optional Volumes []Volume `json:"volumes,omitempty"` // NetworkNamespace defines the network namespace to run the container in. This enables access // to the host network namespace. // See https://man7.org/linux/man-pages/man7/namespaces.7.html. - //+optional + // +optional NetworkNamespace string `json:"networkNamespace,omitempty"` } @@ -350,21 +351,21 @@ type WorkflowSpec struct { // TemplateData is user defined data that is injected during template rendering. The data // structure should be marshalable. - //+optional + // +optional TemplateData map[string]any `json:"templateData,omitempty"` // Timeout defines the time the workflow has to complete. The timer begins when the first action // is requested. When set to 0, no timeout is applied. - //+kubebuilder:default=0 - //+kubebuilder:validation:Minimum=0 + // +kubebuilder:default=0 + // +kubebuilder:validation:Minimum=0 Timeout time.Duration `json:"timeout,omitempty"` } type WorkflowStatus struct { - // Actions is the map of rendered actions and their status'. - Actions map[string]ActionStatus `json:"actions,omitempty"` + // Actions is a list of action states. + Actions []ActionStatus `json:"actions,omitempty"` - // StartedAt is the time the first action was requested. Nil indicates the workflow has not + // StartedAt is the time the first action was requested. Nil indicates the Workflow has not // started. StartedAt *metav1.Time `json:"startedAt,omitempty"` @@ -388,7 +389,7 @@ type ActionStatus struct { // Rendered is the rendered action. Rendered Action `json:"rendered,omitempty"` - // StartedAt is the time the action was requested. Nil indicates the action has not started. + // StartedAt is the time the action was requested. Nil indicates the Action has not started. StartedAt *metav1.Time `json:"startedAt,omitempty"` // State describes the current state of the action. @@ -427,11 +428,11 @@ const ( ### Workflow state machine -![Workflow state machine](https://raw.githubusercontent.com/tinkerbell/roadmap/latest/design/14_tinkerbell_crd_refactor/workflow_state_machine.png) +![Workflow state machine](https://raw.githubusercontent.com/chrisdoherty4/tinkerbell-roadmap/feat/crd-refactor/design/images/tinkerbell_crd_refactor/workflow_state_machine.png) ### Action state machine -![Action state machine](https://raw.githubusercontent.com/tinkerbell/roadmap/latest/design/14_tinkerbell_crd_refactor/workflow_state_machine.png) +![Action state machine](https://raw.githubusercontent.com/chrisdoherty4/tinkerbell-roadmap/feat/crd-refactor/design/images/tinkerbell_crd_refactor/action_state_machine.png) ### Resource validation @@ -439,25 +440,18 @@ Each CRD will leverage [CEL](https://kubernetes.io/blog/2022/09/23/crd-validatio ### Data and functions available during template rendering -Templates will have access to a subset of `Hardware` data when they are rendered. Injecting `Hardware` data was outlined in a [previous proposal](https://github.com/tinkerbell/proposals/tree/e24b19a628c6b1ecaafd566667155ca5d6fd6f70/proposals/0028). The existing implementation only injects disk that will, based on this proposal, be sourced from the `.Hardware.StorageDevices` list. - -The previous proposal did not outline a set of custom functions injected into templates. The custom functions are detailed in [`template_funcs.go`](https://github.com/tinkerbell/tink/blob/main/internal/workflow/template_funcs.go) and include: - -* `contains`: determine if a string contains a substring. -* `hasPrefix`: determine if a string has a prefix. -* `hasSuffix`: determine if a string has a suffix. -* `formatPartition`: formats a string representing a device path with a specific partition. For example, `{{ formatPartition "/dev/sda" 1 }}` will result in `/dev/sda1`. +Templates will have access to a subset of `Hardware` data when they are rendered. Injecting `Hardware` data was outlined in a [previous proposal](https://github.com/tinkerbell/proposals/tree/e24b19a628c6b1ecaafd566667155ca5d6fd6f70/proposals/0028). The existing implementation only injects disk that will, based on this proposal, be sourced from the `.Hardware.StorageDevices` list. This will continue to be available for new templates. -These functions will continue to be available during template rendering. +All existing, custom template functions detailed in [`template_funcs.go`](https://github.com/tinkerbell/tink/blob/main/internal/workflow/template_funcs.go) will continue to be available during template rendering. -### Hegel Changes +### Hegel API Changes Hegel will undergo a reduction in the endpoints it serves because the data is no longer available on the `Hardware` resource. Minimally, Hegel will serve the following endpoints: * `/2009-04-04/meta-data/instance-id` * `/2009-04-04/user-data` -This ensures compatability with the cloud-init that explores the API via the `instance-id` endpoint. +This ensures compatability with cloud-init that explores the API via the instance ID endpoint. ## Migrating from v1alpha1 to v1alpha2 @@ -501,20 +495,24 @@ Workflows operate as a state machine (detailed in [Workflow state transition](#w For these reasons, we think it would be more appropriate to have explicit fields to represent state and consider a separate conditions proposal that compliments the CRDs. +### `ActionState` and `WorkflowState` + +Having 2 separate types helps decouple the Action and Workflow state machines. This ensures adding or removing states from Actions or Workflows will not impact the other. + ## Implementation Plan 1. All services that have unreleased changes will be released under a new version. This provides a baseline of functional services that can be used in the Tinkerbell Helm charts. 2. Develop the code to leverage the `v1alpha2` API. -3. Hard cut over to the new API version. This means versions beyond those specified during (1) will no longer support the `v1alpha1` API. +3. Hard cut over to the new API version. This means service versions beyond those specified during (1) will no longer support the `v1alpha1` API and consumers will be forced to convert their resources. ## Future Work -Introducing mechanisms to propagate reasons and user readable messages for action failure. As these mechanisms do not exist currently and will not be realized through this proposal, the `.Workflow.Status.Reason` and `.Workflow.Status.Message` will have minimal benefit to the end user. The separate proposal will address the contract between actions and Tink Worker that can be used to propogate a reason and message to workflows. +Introducing mechanisms to propagate `Reason` and `Message` values for action failure. As these mechanisms do not exist currently and will not be realized through this proposal, the `.Workflow.Status.Reason` and `.Workflow.Status.Message` will have minimal benefit to the end user. The separate proposal will address the contract between actions and Tink Worker that can be used to propogate a reason and message to workflows. Maintainers have informally discussed inverting the relationship between `Hardware` and Rufio CRDs. This is necessary for defining a precedent on how to extend Tinkerbell functionality without expanding the scope of core Tinkerbell CRDs. Introduction of user defined metadata to be served by Hegel that could facilitate user defined actions. Similarly, injection of additional `Hardware` data when templates are rendered will be addressed on a separate ad-hoc basis. -Separation of `Hardware.Instance` data into a separate CRD possibly owned by Hegel. Given the instance data is unused by the core Tinkerbell stack it +Separation of `Hardware.Instance` data into a separate CRD possibly owned by Hegel. Given the instance data is unused by the core Tinkerbell stack we anticipate, in conjunction with adjusting the relationship between Rufio and Tink CRDs, that instance metadata will be deprecated on this API and transitioned to a separate CRD. From 66e3a12b39bbebf83f4035e8ab5792bc78e947d3 Mon Sep 17 00:00:00 2001 From: Chris Doherty Date: Mon, 27 Feb 2023 08:30:14 -0600 Subject: [PATCH 06/14] Update ToC Signed-off-by: Chris Doherty --- design/20230222_tinkerbell_crd_refactor.md | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/design/20230222_tinkerbell_crd_refactor.md b/design/20230222_tinkerbell_crd_refactor.md index 727b74f..928d4f2 100644 --- a/design/20230222_tinkerbell_crd_refactor.md +++ b/design/20230222_tinkerbell_crd_refactor.md @@ -13,14 +13,16 @@ - [`OSIE`](#osie) - [`Template`](#template) - [`Workflow`](#workflow) - - [Workflow state transition](#workflow-state-transition) - - [Webhooks](#webhooks) + - [Workflow state machine](#workflow-state-machine) + - [Action state machine](#action-state-machine) + - [Resource validation](#resource-validation) - [Data and functions available during template rendering](#data-and-functions-available-during-template-rendering) - - [Hegel Changes](#hegel-changes) + - [Hegel API Changes](#hegel-api-changes) - [Migrating from v1alpha1 to v1alpha2](#migrating-from-v1alpha1-to-v1alpha2) - [Rationale](#rationale) - [Comparison with existing resources](#comparison-with-existing-resources) - [Use of explicit `State`, `Reason` and `Message` fields vs conditions](#use-of-explicit-state-reason-and-message-fields-vs-conditions) + - [`ActionState` and `WorkflowState`](#actionstate-and-workflowstate) - [Implementation Plan](#implementation-plan) - [Future Work](#future-work) From 4bd09ed46f7ddae057e6b2bebb0a6e5c8a39e396 Mon Sep 17 00:00:00 2001 From: Chris Doherty Date: Mon, 27 Feb 2023 17:10:35 -0600 Subject: [PATCH 07/14] Update to use conditions Signed-off-by: Chris Doherty --- design/20230222_tinkerbell_crd_refactor.md | 137 ++++++++++++++---- .../tinkerbell_crd_refactor/action_state.puml | 4 - .../action_state_machine.png | Bin 13184 -> 12057 bytes 3 files changed, 107 insertions(+), 34 deletions(-) diff --git a/design/20230222_tinkerbell_crd_refactor.md b/design/20230222_tinkerbell_crd_refactor.md index 928d4f2..f205b30 100644 --- a/design/20230222_tinkerbell_crd_refactor.md +++ b/design/20230222_tinkerbell_crd_refactor.md @@ -133,17 +133,18 @@ type NetworkInterface struct { DisableNetboot bool `json:"disableNetboot,omitempty"` } -// DHCP describes basic network configuration to be served in DHCP offers. +// DHCP describes basic network configuration to be served in DHCP OFFER responses. It can be +// considered a DHCP reservation. type DHCP struct { - // IP is an IPv4 address. + // IP is an IPv4 address to serve. // +kubebuilder:validation:Pattern="(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)(\.(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)){3}" IP string `json:"ip,omitempty"` - // Netmask is an IPv4 netmask. + // Netmask is an IPv4 netmask to serve. // +kubebuilder+validation:Pattern="^(255)\.(0|128|192|224|240|248|252|254|255)\.(0|128|192|224|240|248|252|254|255)\.(0|128|192|224|240|248|252|254|255)" Netmask string `json:"netmask,omitempty"` - // Gateway is the default gateway address. + // Gateway is the default gateway address to serve. // +kubebuilder:validation:Pattern="(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)(\.(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)){3}" // +optional Gateway string `json:"gateway,omitempty"` @@ -157,12 +158,15 @@ type DHCP struct { // +optional VLANID string `json:"vlanId,omitempty"` + // Nameservers to serve. // +optional Nameservers []Nameserver `json:"nameservers,omitempty"` + // Timeservers to serve. // +optional Timeservers []Timeserver `json:"timeservers,omitempty"` + // LeaseTime to serve. // 24h default. // +kubebuilder:default=86400 // +kubebuilder:validation:Minimum=0 @@ -376,14 +380,8 @@ type WorkflowStatus struct { // enter a WorkflowStateFailed if 1 or more Actions fails. State WorkflowState `json:"state,omitempty"` - // Reason is a short CamelCase word or phrase describing why the Workflow entered - // WorkflowStateFailed. It is not relevant when the State field is not WorkflowStateFailed. - Reason string `json:"reason,omitempty"` - - // Message is a free-form user friendly message describing why the Workflow entered the - // WorkflowStateFailed state. Typically, this is an elaboration on the Reason. It is not - // relevant when the State field is not WorkflowStateFailed. - Message string `json:"message,omitempty"` + // Conditions details a set of observations about the Workflow. + Conditions Conditions } // ActionStatus describes status information about an action. @@ -392,7 +390,7 @@ type ActionStatus struct { Rendered Action `json:"rendered,omitempty"` // StartedAt is the time the action was requested. Nil indicates the Action has not started. - StartedAt *metav1.Time `json:"startedAt,omitempty"` + StartedAt metav1.Time `json:"startedAt,omitempty"` // State describes the current state of the action. State ActionState `json:"state,omitempty"` @@ -411,9 +409,17 @@ type ActionStatus struct { type WorkflowState string const ( + // WorkflowStatePending indicates the workflow is in a pending state. WorkflowStatePending WorkflowState = "Pending" + + // WorkflowStateRunning indicates the first Action has been requested and the Workflow is in + // progress. WorkflowStateRunning WorkflowState = "Running" + + // WorkflowStateSucceeded indicates all Workflow actions have successfully completed. WorkflowStateSucceeded WorkflowState = "Succeeded" + + // WorkflowStateFailed indicates an Action entered a failure state. WorkflowStateFailed WorkflowState = "Failed" ) @@ -421,13 +427,96 @@ const ( type ActionState string const ( + // ActionStatePending indicates an Action is awaiting execution. ActionStatePending ActionState = "Pending" + + // ActionStateRunning indicates an Action has begun execution. ActionStateRunning ActionState = "Running" + + // ActionStateSucceeded indicates an Action completed execution successfully. ActionStateSucceeded ActionState = "Succeeded" + + // ActionStatFailed indicates an Action failed to execute. Users may inspect the associated + // Workflow resource to gain deeper insights into why the action failed. ActionStateFailed ActionState = "Failed" ) ``` +A `Started` condition will be used to indicate the workflow has started and is observed based on the presence of `Workflow.WorkflowStatus.StartedAt`. It's severity will be `ConditionSeverityInfo` and its default Status will be `ConditionStatusFalse`. + +A `Succeeded` condition will be used to indicate the workflow succeeded indicated by a status of `ConditionStatusTrue`. When an action fails, `Succeeded` will become `ConditionStatusFalse` with severity `ConditionSeverityError` and the `Reason` and `Message` will be propagated from the failed `ActionStatus`. + +#### Conditions + +The conditions data structure will only be available on the `Workflow` CRD but is designed to be applicable to other resources. It provides the foundational components for extensible observations about any resource that it resides on. + +```go +// ConditionType +type ConditionType string + +// ConditionSeverity expresses the severity of a Condition Type failing. +type ConditionSeverity string + +const ( + // ConditionSeverityError indicates the condition should be treated as an error. + ConditionSeverityError ConditionSeverity = "Error" + + // ConditionSeverityWarning indicates the condition shouldbe investigated but the system may + // continue to work. + ConditionSeverityWarning ConditionSeverity = "Warning" + + // ConditionSeverityInfo indicates the condition is informational only. + ConditionSeverityInfo ConditionSeverity = "Info" + + // ConditionSeverityNone indicates the condition has no severity. + ConditionSeverityNone ConditionSeverity = "" +) + +// ConditionStatus expresses the current state of the condition. +type ConditionStatus string + +const ( + // ConditionStatusUnknown is the default status and indicates the condition cannot be + // evaluated to either ConditionStatusTrue or ConditionStatusFalse. + ConditionStatusUnknown ConditionStatus = "Unknown" + + // ConditionStatusTrue indicates the condition has been evaluated as true. + ConditionStatusTrue ConditionStatus = "True" + + // ConditionStatusFalse indiciates the condition has been evaluated as false. + ConditionStatusFalse ConditionStatus = "False" +) + +// Condition defines an observation on a resource that is generally attainable by inspecting +// other status fields. +type Condition struct { + // Type of condition. + Type ConditionType `json:"type"` + + // Status of the condition. + Status ConditionStatus `json:"status"` + + // Severity with which to treat failures of this type of condition. + // +kubebuilder:default="Error" + // +optional + Severity ConditionSeverity `json:"severity,omitempty"` + + // LastTransitionTime is the last time the condition transitioned from one status to another. + LastTransitionTime metav1.Time `json:"lastTransitionTime"` + + // Reason for the conditions last transition. + // +optional + Reason string `json:"reason,omitempty"` + + // Message is a human readable message indicating details about the last transition. + // +optional + Message string `json:"message,omitempty"` +} + +// Conditions define a list of observations of a particular resource. +type Conditions []Condition +``` + ### Workflow state machine ![Workflow state machine](https://raw.githubusercontent.com/chrisdoherty4/tinkerbell-roadmap/feat/crd-refactor/design/images/tinkerbell_crd_refactor/workflow_state_machine.png) @@ -457,7 +546,9 @@ This ensures compatability with cloud-init that explores the API via the instanc ## Migrating from v1alpha1 to v1alpha2 -Given the relative immaturity of the Tinkerbell API we will not provide any migration tooling from `v1alpha1` to `v1alpha2`. Users will be required to manually convert `Hardware` and `Template` resources. `Workflow` resources are considered ephemeral and can generally be deleted. +We will provide a basic utility for converting v1alpha1 `Hardware` and `Template` resources to the v1alpha2 version. For fields that cannot be represented in the v1alpha2 API the data will be dropped. + +`Workflow`s are considered ephemeral and should be recreated by the end user. ## Rationale @@ -491,16 +582,6 @@ Reasons for failures have been introduced as a core concept, via the `Reason` fi The `Workflow` provides a `TemplateData` field that can be used to inject arbitrary data into templates. This facilitates users wanting to model variable data on `Template`s that has per-`Workflow` values. -### Use of explicit `State`, `Reason` and `Message` fields vs conditions - -Workflows operate as a state machine (detailed in [Workflow state transition](#workflow-state-transition)). [Conditions represent observations](https://github.com/kubernetes/community/blob/master/contributors/devel/sig-architecture/api-conventions.md#typical-status-properties) about a resource and are not state machines in and of themeselves. Conditions should generally be complimentary to existing resource information, not replace it. Conditions in the Kubernetes API have a [well defined set of fields](https://github.com/kubernetes/community/blob/4c9ef2d/contributors/devel/sig-architecture/api-conventions.md#typical-status-properties). Some tools such as [Cluster API](https://github.com/kubernetes-sigs/cluster-api/blob/main/docs/proposals/20200506-conditions.md) break away by defining their own condition expectations. - -For these reasons, we think it would be more appropriate to have explicit fields to represent state and consider a separate conditions proposal that compliments the CRDs. - -### `ActionState` and `WorkflowState` - -Having 2 separate types helps decouple the Action and Workflow state machines. This ensures adding or removing states from Actions or Workflows will not impact the other. - ## Implementation Plan 1. All services that have unreleased changes will be released under a new version. This provides a baseline of functional services that can be used in the Tinkerbell Helm charts. @@ -511,10 +592,6 @@ Having 2 separate types helps decouple the Action and Workflow state machines. T ## Future Work -Introducing mechanisms to propagate `Reason` and `Message` values for action failure. As these mechanisms do not exist currently and will not be realized through this proposal, the `.Workflow.Status.Reason` and `.Workflow.Status.Message` will have minimal benefit to the end user. The separate proposal will address the contract between actions and Tink Worker that can be used to propogate a reason and message to workflows. - -Maintainers have informally discussed inverting the relationship between `Hardware` and Rufio CRDs. This is necessary for defining a precedent on how to extend Tinkerbell functionality without expanding the scope of core Tinkerbell CRDs. - -Introduction of user defined metadata to be served by Hegel that could facilitate user defined actions. Similarly, injection of additional `Hardware` data when templates are rendered will be addressed on a separate ad-hoc basis. +Introducing mechanisms to propagate `Reason` and `Message` values for action failure. As these mechanisms do not exist currently and will not be realized through this proposal, the `.ActionStatus.Reason` and `.ActionStatus.Message` will have minimal benefit to the end user. A separate proposal will address the contract between actions and Tink Worker that can be used to propogate a reason and message to workflows. -Separation of `Hardware.Instance` data into a separate CRD possibly owned by Hegel. Given the instance data is unused by the core Tinkerbell stack we anticipate, in conjunction with adjusting the relationship between Rufio and Tink CRDs, that instance metadata will be deprecated on this API and transitioned to a separate CRD. +Separation of `Hardware.Instance` data into a separate CRD. Given the instance data is unused by the core Tinkerbell stack we anticipate, in conjunction with adjusting the relationship between Rufio and Tink CRDs, that instance metadata will be deprecated and transitioned to a separate CRD. diff --git a/design/images/tinkerbell_crd_refactor/action_state.puml b/design/images/tinkerbell_crd_refactor/action_state.puml index 1c3e0fa..38aed03 100644 --- a/design/images/tinkerbell_crd_refactor/action_state.puml +++ b/design/images/tinkerbell_crd_refactor/action_state.puml @@ -7,8 +7,4 @@ ActionStateRunning --> ActionStateFailed ActionStateSucceeded --> [*] ActionStateFailed --> [*] -ActionStatePending : Action waiting to start -ActionStateSucceeded : All actions completed\nsuccessfully -ActionStateFailed : An action failed - @enduml \ No newline at end of file diff --git a/design/images/tinkerbell_crd_refactor/action_state_machine.png b/design/images/tinkerbell_crd_refactor/action_state_machine.png index a370c8c96361bc671dd90614a6b72e71252ea4ba..ca1cb70d998f48c817d322ccdd187bcfbab3ec48 100644 GIT binary patch literal 12057 zcmch7cT|(jw!W1bdJRffO6UOu1VOq$q(-C(1PIcLAc9h)MWideD7^`2kluUmAc#}} zDbkyC@Hg>0=brD}d+u7lzpiV!Aj4$ddH3wu`+1(dUu&u>lMpcwUAS<8MCG1>_Js=< zEy4dvf=gfo&P!MgesH@g>baUbI(gWkEnP1tqaDysP_AeT7BdePD_2*i+X4bkb|?qe zr}lRI=8pChf)dxk3ar*hJ=cGZU$_XCaZe75R8wmaqz*iwJ7(VaOmn;B0kSF*x|9at zgJcU|d&64xmBETiCCcyjbjkeX)wTtRCCSOC9?9m?B76TD5xLHcVN5umfJxvDGAVbd z7fpgSjji#;tkZ7_0t)UNuHGNS=UJdW+so!EMynvTelfc@D+;sLpRli5{#MlpOzH5i zG+blaxVXl^-z(jZ<(#U#)K+{$_3gKq6v-m2C!`jmHLb#&tb zVwOW```yIp8)6TxPr0%is5)zUzrpVqHR1S(7ASekr*e5h-rH}>7`mU`LttVYP~ODC zzHT|o`iiF3X@dSC-^kWi?v0T9?j+E-(uGn<1lB^7Th@@1$h7gG|gnVY3)%33l#=L&=*e>_yNnP@4<%|9sop-;Jd^TEWxU{~0XY5M;!%uZjS7|RRqRDqQ z8w2a>e}6Bdm+|&Qqi;5YvEy`m6M|4iN!kUd2+Y4iU^x5!wMiz;^9Y%9)5%=OTzD?> zL*qK9fIviA20Tb|XVlP9KP~h%6(`&kiRUc&(YW^Zn8B7YUSt=`Mxa4qnE}*Y;PO2* z`GMI)nY}L0d!-wc))mn4G9te8~*L{zfWmo0rUU+SN=7{|1O&R zKd$;`%K!a?=P197jwWhhrt^XMHp1o7xVX86K7@Y$4A0W;B-erloYEEQ65sbcKU#X; z^->jUWMb0)XP8V{S{f@aE9;vJDK!WStn)g7ivznHC(HOuXVB2eiafb!G0Hp_x0+FuCr4+$Lg8u%=i2w0jFw!&l<%@_#2f2X2Cs}_yRKi7YMArT- zEefIVx%qi05zIfbEbr5&SMZgU6?*WAJVgBI3N`KouI((P_}GXfCMPHVTK&p-^QQd` zN)`zT58$@?`uYz)8li%X`rqD1I}0-BhwLx1_qh4_jXzru%HXBLuPZq?@GH@-1*^A&T@w?W{7&bR z2f=5F>()pXZ&Y7)6}fe*iQ&4fCe%%_P?uTaNx)RUM>~UB96y>t2kJI={A(2$W6izK z`_X*MCrACAwAb@SzpL#L%wh&_0#D3?38^^t)+gvl z!$Lznw_E9)v9CxOSDesNW+FY4pWeKC*BcFX-kp=;ZKHB~y7tAhhL;zcJf!*g`;buQ zqj%4yp+aT~NfRYoBTY?mUy`Dt%=LP#6rULurPD_XJ>RoJd349}<+_f1cwpI8?jb&> z`UtH@N3M3)7M_&{nRxs5t(gix)1BwkXOB;eW5WGX{tgn)HYTlyvr)yI8tsZ|0j*lBelMxeBjsD*07SO~a3TZWs5~x3=?AE*tGbpwU z+Zr0lf0%tI+CZm{W8sT;1KCH%bMuhqv4?rY1!FIVw&_{m;bS?&_~s7tTE;G6S~vEuU;u1 zY)lEwHK-|vUvE?^Dk`Fd$IR#oJG-7xE=qH8Pk!Z6|0T5fvc@!Fa?&V+28EY~ zgb7hU-|ZC(^t#2%u_hdDX2Ih$@bcni1$1hD{pn73{FL2v4b+8e7kB(^qIzw#p~mN5 zbTET_)HF1xqvC#P;i{`J|E|aqIEQ-jSm(=v0^SgDO?4|?p~#>dA@h={(U|^J{o@QjhXrx z&)>epv;xLPy1Kd#AC^B|)h%c@CTw-(IbKY1#GcN+VrG(Z7Zew#@7`VP73RhGJnl`r zy6OeO+Y$(j&T|3c*9AX7gco|nH-3FJX-9oDsrPOVXQWq3s64f{AtvWJZ{>ohu7CZ+ z99wi5i5J8A6b<4XB2h%e5$)|Q4HuV?fOdcV_~^yK)H4)$fyYW-4kZ;;(dQ2zR5_wh zVS@eyL_|9W2XxbhhK7kkNa@f z-t0L|Z@f^38qeV1pkw{ntx?BspIfo?Efi4bYcsv!Qrm~l@&4ms)6+H|fPwKqh~1p3 z78{j1Su6he^XF96Q>vJ>6MkXC268(Pfx+D+AN^|L6GrgKxHb`7zU1!)m_U;YvgKb+ z_VMw4n9YpNgIt4I=$%g#*#ujywTNdD9fcoQTTeOCpUW@zl_<1kbANK1>=Xuc$1 zKel}k#Wp%NcDOv-+t-J=4;(`tU^D)7$k1#{d%H%8G^23E(eBd0+Bd4=Bom*r81PVl zt#*pnx3;zbB&*%&;)xF#`SPXYh!!|{Dgbb}2{^0;EDV4b7-Rvz%^uhMK0oPfdGJ>d z4~G*d|N8`496jQ<^RJ3o3R2_+(|p1~=0sHI&rDN!pp$cC-M6%gEn{vefL) z@8Z1?rxP&l>FUCzX*aezcNm3HZc8O>XYjjQuLyTwGiN0*`T7 zUNL`!gM(uy9W$(@t6TQsz=12QK00fR!?#IaJ_xc6`*j3#zK8@*KfB@PN zBmoEw9tp!0G=yI8LN{*c-n9aQQlc)sDpziK;QK?je*OBTsr2X(Bkq$K1%O{>DRz6t(Fesq}iD^Crv*H9y&gUa!K5N`2Zf$KcAg^didkX5ny;@XUWmv9_jg;lfatQb4OO#mBX>~5+{28V(&G<_4xSF{P0}#p-fQ$Bi4inSz zihG|{3*X`)`#>C^C-vNeX>(D>Yf2qVx(L+n4F%KiAZTo@tQtl& z6(gAL2wFCBwP`|CvnXRWrm7!C_klYRxRBy=4o#w@v9u}Y^T_Kp6LXwy$?uo>?Fn*_ zlL8kOz}MZdz*}fD9Vn@;l7+>M=TiXWre|afzS%pL7_B*&egTVtiHg#&`@MZjnSj76 zwy?0cF6B;+o_3!n*;6X=jN^YKxC1z*GQRs-QM=$2xv`uO0)HS3m)7maMeOzg3U5*~ zZx3&+2W+I)^XRi)DN0dGOAFwD?*4ul* zSff)XSkeYNi$XW}2;`C_GP+3^Z+|)UlwD(5w&P-^rR6C2y%1-T3ZbSAY2+~RKDrs3 zhKDS%>}8nE0s@PA0{p7&Kv=7AeF9X|!$X3onF7-q;B|gDAE8K>$`jn>zW?p9mzS5; z1^F0&HJ$wzV^B8bz4e*-kc?Iv8ynZ=z(MjhRoqUVqSwYtiB)RsC(3qorKF?)+Y6__ zH@^Mk2Z3>LSXikhwCRd+xi&RbSjI$%;DM~lY`Bv1+XxT*A6L7B-syj0>k$`BYtUTo z6`P^n|Avjn_BXlQy?FhGRH2$?TZlkikOZ-A_%$*0WNb+uM5o$*RxLXi5XnE_mFn4i zCud3~$iaC$=5sBt-$6RMx|-BpkukUJd`5TkA~8;^ItPht>OX-XP_6*I1ZZYM0_{x_ z9veHm7WXt9e3do*h{C~F03{xslZ$IPn(Z7drDXK1jN~)sLr@4#rmHyY>iY+~-uf4I zU1MOPP#XLfb~P+l!(mr1(|=&s$o~bqZW-6yOEZLIc>NQ*UPYTK?2g-hdb&RT3*7i7 zHE|{9^Go_;%22%xz&_mFE4C=xbZcT|(f1zx4Zk}66TdRC7yb|YT53>e9M=$?252CV z-mA8nCH7|^Llk)ep#5ydiw>_bG3Whr zn2-)Xzy}9pH74jq=lAO z@B7jcey`WrudhMH8?LVbOw4rWYi<&FnQ92Qa+z|+nUA@-4JO@WREAF950pOx&=+;*#Xf+Vm|-6Ov5mT& zpTjxkOkzIgr+~HYZ_fM%SUKwtgp5B8kd%`0{F1Bnsb+`B(n3^J)V?n8511Vy{V$aI z7xYZU>4WGBI8F_MJM#@>BkW-Q8IUF|0BHZ9)W0U+q`|-c7C#2=>%Xw+->2fB?O#&` zi}E0bX(uJda}0K4Sl`zGn+AieB6W2?Csjg#Ue(#rVG1H?S$a{C)?p#b9K&^K&x)$5 zMBGeruFG?#k6lPc3-p`+C@eq<7(ca!%u$1@A;iNIf`hurdwZ@uzTXy--_h{sO1W>n zJVJoM*x1<2%o}bJIoYYHsWmmponVl#P&X+)z9WP|T~%NI8wmIbZAeUQVUhRQ^FGbw z@cH(L?u921nuKNm3oN;S3_(BlnfW#`PG;fdwOG-HhG$k)G0bUlQ8R=so}Zmi{=5d* zlgq^T)5>R?#z40MiMY79cu?u-)2BUpFnLO#k}WPQWC84=`VwrN6biSPZwn*MWBirA z4~TEJDxL@lLy*2IntVQkiJ6(X)L^~Aiwg#8l13ttSczN+n>7OigQu6*?(QyQ9VaZN ztG_>D$qnXzFAgXq0T_(gb^bSJuYpiNO6uU?U^%P_RR~s4R&H-=Q(c*tU(*K3NoFt; z9`vJu`wk$kHqY+L)6yTdw6+d1Ao1Ri1C1{Jl8C6Niqj1CAuB5@aBDjbu==GKRT_Hw zv4H`WZ+E`fl-AYNDJm-3@xuJY>G$MiWa^KPpTFLG{NTX|7S-I<)iq`TRBCz$Ot?klX<|ZVQnBx$ zS0NN#?yd#>p*3hz8o!>Cl~sBA<^vLTJ#Vj`R~}9NbLV@b|M%~v2S*6mJS5)LtJ~bh zV3!!+f*l?Ao>jZB3JD2`h(wP9vRLiQ3xL8DcDJRiZEh^#0us*}ADjYxy4o(cp<8@- z;Hy`XCyd+R>&!%fY=VLV?*y(8MW{ZT-kj2cws=$1(VZM0@AzMy%}B4Qk;=XjBa6W8 zdueHjSL54fe^JwZ02qX*k$6Iv9Q{PD4BkZ?uF1gs=_NAEJON{9(p=UuFd(vZ%2q;0 z{$`u}TncC#risg@G_n6Y1K#0dWo=DJMy7#4Sa9`3#>Udp(kcOw_s0xbjsZ}H=La}I zz&!PM2y345*;xUgw6oHJjPA+D|eSB3Q9t7%ty%SKsV%`KK%uJsV_yQUsEq^D`Q|`VF7YAW_Aup z{7;OgSWTm66=YEN(d0Z!vjDc#pz|{{^GD)mDu+IafbIEsQczO9jgD^7bY3CR+5V=C~nz~ge_Ej_nQwAFv6_ks%Hn#E5&ze-XjLor>1{>yJ-_sHw#27%cdJ;GS< zWCaLuI3;cC?KNldrmW>607wA>ZweqhWe|#)H8nFciwt3EYf!7r_S8YaJ64N&SXd zpyJ@{yv_5ehAn#@TqmI9HyC0vNp@KDE6cG{G z@(UEcWU{4yg|Q$MfPIh6>DtahyBmFk;|>5vfWQy*kZ3z&`0$M+!dO@a7E5j@z{QnY zP+-T_*AMC^q>Q&qY(_qS#G~@a29>QMMK!M528dfMpP{aiQP20P8_Nk+CWu8weqzf+ z9SgafTkWqvZKT$1o#FDz_>ZNf&GFK1-)0ia#9|9M&fQUEp6Lo_e`i;G^%Ut^uM;=r zFoAgMzPE&n(Y{(lg`2w9?>sNTVh7HJ69V#>|2M%Z`g7LB`3g=Amd;(|# z<2mgO)u`G!(q;se$FGrx%)d<>rfaNcoolz-u2FFyuo@in4?!7e1DCW5o&r%v?mFxD zKOB``|FgoWn6$L_V7j7iI~~#9&JKf&_h}x8UBCdQjeZ~%Un}frInC(3NELA0>tPqUyvtU{t4On zp>Nog6TjUQKC#P%OrZUKb%{UUZw`RfYi!j(@c+6zZ}(!0N0cSLEmCH<+QE!6sy0ceZc%Mf!2y_2%I>#)?FNW;;JR z`|lh~stMGVn6(n=w1VW16-9#CWXUQ{4njI1k@&ky{dBQHRc&r91#l@(9iA{<8_P3r zJmis}E?5UDW8$$NA5{kzILg&4V*Emsv4LXLE+&Ct+LfZkUaZ*)ETIfMof4Cu2~*mkBD5dXVE zwEa@M5K?bYH3NmmW=oCtQrtBnZfkm{l22E@bjJyVJh-L{N&=#e(>1ta%9c~~S1S>v zkXa}4h3IJN&;!9RgqoVL4)NVpV1$XLC-(MyU3Nez{xX|nVEsW`XSIl;WKWsdtK@M% zlZ=dvou=JWo@q0@YHt4oj z<$<(CW#>8B^!hYUJ-vABd7O#2tb96F4`P1`h^!fu1QQJ2iM_`4*UAXDPPQq@jx}&2 z`*1Z-8qlKWdkiWvOP~ip?~AOdY3h*yg1dQnHc8)FFE8&GeRpbSjz$W9fg@mjm1>ij$_M9Vf~PYiIy#M~DViAB10*@QX7uH|Mjv zhZ-Z~4i-p@kH@caZVaNn3E*&^q);=KggdaTzqeP^fTGPhNLpRfvF<1v+mB<^ZCasr zR}*|Yi~oH`;wGrG#oi>6Pp10Yw2!qgCl(Agn=|#lXJ4`8^*D~^!q*}Rbzy&_wBiPT zDl$zNYJLGimlqdJOiWt(-eRaLK$AiA>$e!1#%!!z|NA?$k3t6!qM6u!C+sO}2XePG zUK=ymELJ>(1!pA0(tZrW3PV{HjiWOyx%aRWb3}8m{-|Btpk1z^cATRuX7i)wJmH2U`=zr8|zMBh|0x*k; z4^H6gdqQ5ev7nLh%F#7zP_?(f2M|SyTu{&794oqHRs&ry+zS36YU67XW{^O0uBZZWVz&mmj^=*g)1eRP9f~&Zbk| zXk70-g{?U{v4`>S0FSfXi=~+gcJd~_67e=yuDoXYe6r4~Iz9c858tP(M>vb2vMfjE zYw1nRgKbN+d8XOU$n>Xf4ELB?7H4B0ag2ffAP;ALH2Dil&=6j_zqFJ0JI}0{2;k*+&-ug;nIc>%!6cU9|18N?zDkX&4LLCl;?oi zeKCX_F@r-xpF#4#>zi8q2=JuOH#*Bi7qlFwu>DapnGAu<6O~B|mMV|kMZh3W(a*p$ zq6y#ilrMmCG!8DYRLRE{JgP3Mt2-CK2M>!yEPt^y1hVWIkaQtgwh?k5X)ND05gbnD zqPnd=6bouO@nd?2QN=Bl>f9!t+gBOBvhGq?CJI~1aS~5x=Z|cFmBAeVnrD^IZ)uom zG~X;R4kd!Lgr$$u!@~o#t+<_gD)=REG~Rmqgu~V5q022Xv3%qQq#2Zco&*a2kr5&N zc03%=XfA3Z*S`A9;3>F0$UIi;CDAXkb7>L=ehQeWnQj1Bi>m3>ycan)XI9b&ig>s_ zno$J2qCxtHK$rEFM? zDyGTz$HGDyjjEa&Sf!aT_v1%W0)o4vaB5OnVEUsYz#jq4!VGHy+*^c^zuQA`1d9I{ zzb+?uWU2VqZ%0McdsofEwU|IrL;NPFaDeOR_M#;_DJnJ=V=D1m-Io3Z^JQt#-_J7)c_9ywTHhfvIi7M zCIfarBu3p8Od4vy=UHPF8J9)GYbpKy*LPdQ5gBkR60&mjJG94g;9~X;Mjv$h48^G#%L0=?Bsn*}o>2;CTT-=p+>h8qwlE+wTu8puV3vf#}#GYV%{R}HnaW-;x?4<6i% z>RcuUeQ^;$O5eNwtQgPp;zL?GI^m0;gU$h*j+dvW`n%f;9PJ_Ot=mm8cF$q{dW1?) z@zpASQKwM)2ixtvV5X4y)30Ti@5Bc+Zu02J@SKky`MJ0{9{8|SVX=ZNEH7nr6~M4U z{z$q><@6FkQv~bI>~|#IPxpWn6PK)NR%hPRZ6wes$Y+^2zhc2c(@Q|KN$S@ONKtTJ z;Z$y*1Dq3Q)-_oj9jdoBMX01W%pcYvCXBY|;VI^iopeu~CPRRJ2*Gx0e6N&D$$L2xi z3IM&{L)GFNx3;vncC^jU%|X1g%x-=k#n~$zULGgVGn7xv9fv^LA36vPzdF zb(F5zJp=;cvmFYI7NMK}2&j@6Jiru2)@L)6Bg*F3mg9xYUB%e!+(3&JC_q}~0;Aj( z5ffWlT+;?Qo-dGk|mMyMP5~{h4{bTG-Li0WNug z?9sr$0O+K|b+tM?Pnc)^efX-r-bY=%kJ;xG^+C_rn5hd?v?A;K)zReTi(u(^vt{)= z2wyEc3@d2HvKRnPo^2}aSLFX*KyZ|ckFU3$@}%y#8vi6=5LZV5s^kAo!Wy?$VTFUEQxy!&rm`P^^uD z`3%dJr_TuefepxR-AALdG_~`PfP^Mo1ZI0dK|!%S^=#(#Z+^ovJFGq}zY$k-3%U7t zi8X`;)X+mH*p-%z$M+5n#6E?BeG%Q<<_}e-zYAQ84*PPt-rj7b7Ic|D>NY-SF zYv3Ji)IfkIk6+yE22CE;%rz%bAYw7bABT5#MsW%~ZtO_U&0RM6eH?d4S#J)I#Iu}7 zGZu;}-cd1KEG2CTxX#|48VK%^{g4r+R(s#odiPJKs@nTA&=V~8_de%);lJ^Z%x-nP UdD(#vn$$0-D5@)blr#1Fe=H8Yl>h($ literal 13184 zcmdsei93}4`|mT834?57DTGw^Ewl(tgk<0MEN#|gO@t9r$dVAU?^{&%U1+gysq9Iz z?>lks>GS#i&iP&EoIl{4uFFL;^E}V{eZTMJ^?Kd+Jk?NBq{1GEI4_5{2lv<;Zjxkbz9T)flENv# zdBj~)ZpJNEwK6)tjCsGt{%Eqp=EJqb-izW*EN@Flu7`Ko%!>Rdoop`CV1DzYX7r&f z>!k)dU0xlAH*RDT@w$BB^E97^7%`SlfBATP2v{4EKay|E{wC6;`D-{%A%$u0-sMCN zs+5&c>5h|~>>?Ud9>Nhtdm0~%L%tjGhQ;81>lGYzv(I`$Wtx0ZuIZbWll-h5iqtOT z9F>RCB_$8)gf`!Z0uCkf@bo{IJq81pZ(^=5yzOXpLMaSp;&}p66doF~_uO*!9^2PB zX`|AR*1c|h(k6%ibCr9<@03#|sO%%@7nai-2E8z-!k#c3Gi+34hzx-~iQSNy}DKjNYb zQlg-s`uU2iDdL_*Agf&f!-P^Zx z0f#V%sPp1T&HP~5#q;Mwx*MYfjTeS0{2o0*6V9{mM{=tl@1=`8t;#1Xtmt?dg%4Wp ze0kpSgkYGgADZC)@{-T~%YJ<8!QN^o&p96CiEmq5n`i;Igam^;Dco7abHk1cCvxKh zC5{b;8j+x^B@N23=Itx_zC^;C|Oi}<@Y+Dx^8T;t0qAM)(k7}qCqr} zB84SM8Z4qkYKE0X;d?kF$cX?b57EQ@i%)1S4pTs!0=u($rz zEUi2l;gB_m;Y2UjS@)iHIf}+0%jd9Nj?6fu-sjDmH$076uIW|Ncv` z8v+`36PNS)HKPWhKMVF~$t5K8Iysr3k7{mie){w&&k3+qY-Ob+cGusZL=cmnmDTHF zZ!Z`G#yxRoZ)+Jh|KP!cS7@2tIk7lqQA-jmDHa*eN>6XfJo*;-ld;7v>B6I`A%_sX zM@huCKi{J9arccR={+UX@#DvN7>|Nc_UqljQhPXP z1_lNVMC1jmr_$lfnKLb|tzYVQD%|Fc)YLjA+vBN^OYt-;%A^qVmX>}ruFrn`{rfkU z-4*f+39lD3lw*>Sdb3q(a`FntpND1Thrm?ar%owlNg^2LoxQBB!oosD|Es>MzdPB_ z-vd)0P=BFb-iyJ3{l0(yu7PZg?p`WBIQ6O4Yd5`19*gTbV_a%?S&GKj*SAkCx?_8! z>b^RbE#opuunwc5to$yOWwV*dY_3P8@Olj^E9;2Y=J3XVQ)i;|*L=gqOlc{pp`oEG z4k#_cln*-!Q;ffMttEe6ll`?`_4_k@yxiPb!wfXXj#a{L zyZH%1UN$y-I1cL*TwDdSuTNp5cIV#Nc|h=|6WZg?IVZ_jDU4QDvpb4OFm&Y#8VH9S zmktUIeGKkotBb#W{W$N~eA#H`^kA8TuvzWHdIIXt&bo1l4TFTs_cN{0f~)K6?Vmm+ zmrV60Cv)7@zP~V(qQ1Mic>C^{jzg@a{7SnS5fjwX&M(P79R5EZt3ktm-y1Ki^(s{&fhALoj$$rWO*g@a@fw zI~}hoY`?y)y1&@drk$mxd2`^~Hx;YQ`->xK3eS3|Cc!MdmhorpQ>P)%#$^gF)gI~7 zLKYdFoSm&b;Iylai9yU$KS6~Gg%qqM-os_OeNjnC={@RIWTag0V%nz0>#CL94p-IJauXL@P?9x#_OxIfSWxt0HAAVEh z<>hr}A})AR6M1v<^YhoHRj-Z97RH-{?{p^qI=;Are(64)5wo*C%S1~{^eJOwW8bg7 zJEcV~(^@ zEOz?U;(Lfx%Xa)DyN^H7ig>=0r%r8eZJob(@e&lO))?W!+}yX7+J8-q?}6n*wIS}} zZ{{Qx0S&^0L8XiJ$#9YV{e2C@28EPBaUd28_#pAGzcm9cA^t@y9sjv_nm0K=KOauM zx~|QEmf~oT;+V+EGX^wp8FL|lish_}&E)$43c(SD<>lpuhK8X!#xuCENsE_jWF=?s zMynpJIRPJC;sM)`f~R<_eCAd{#c|!S5^PlMj6df9x%qur+|qd|d-e=PcUDfh(5|V* zt};it(##Ud#Z1bhCqCEH`=be$vF2%Otmp5ULy&^Q*Z3UBY<3S<-ItSXrOuJGEzaFt@%@t+@_85dhc?1afJcdeX^_|X{Hi|&-(d^Ne{gHBt!r{S6~n=O3d zLaf_zn9rUcxLnGARL+a0gA`?S@mj!U`2U5p0rNF4L!vioU%xsrwH{4LdN zk6-iis)-p1j{^f&?_S&L1c z+qZj$BNGx#H8jS?$7M60JbNbXIPSj__Q$?wWw_GS`0dTlU?N_B$@>c?1_qyG=$JDU zS97Mu#_BU6D=4?-U$2SF^x0ch#8USAmetiA7(3KNB_=NX{3%DFqo?OIcqfqap}#$= zz}5Rd(OLQAY>52a^3RXf@_Z^Is^qT*=+&i`+r&CG_96>i^@(2elK~$lgC(!@`%-s5 zNa|yBwfradAvP9Sy|cI0(+>Sk>`%GVs&cNehUKjOexTRk) zW~HSWhcJX_hVU0JniM|q*njx&;jv|y4av+X>AXd!g&eN_7*11Db4an6`3{Y5N5usp zvs#taoQmbo+(})k>hw5G2?+^s_uu_BUb}9VmM3%#Znwu}Tqq6vfwh7Yieq7txusfa z_gz+5>xRwd!mx?9^6Yu43liYrR`iuz6rC?c(;|~krEBrBl)%uY;PnZItd z74J}+;*%q&+6!T3B5~bW#X=RcbUdX}`Y=V$Q27$?tgPqq`g#Uk=4rqFOiC$<5qj}( z2!qQgt-ysje*M??Hk&r)2bEAmw%m_nVwl>~tHLvVP|Iv3>tfG4j+5j)EjnZXRZ=yQ>|0in&;Er! zyB}B_x-1hD6U9~6F~jo|tQLc$6g>mw^}M#;k&0h-sW0s<4La0iG(t>?*xFd%ZK>D( zDSh}*in6){Ef2oOq|RGfQ1Dq5H&3yYlvMHE7i()Sl{E%lKR!HTpVK9_c;xQxz- zi@z{7?@pB;j!Q|76T5T7tMdNh^@9JlL_YJmZ*TIMxm8})tK?Sk%BelOFMmi$MJ2sX zgU}~xNhuISRrykA*n;RY#pS}Bu zw|6DqhO3D8kb3(Q`s(|O$9eH0PgJnSP$WV8c`YvdoX=tld>*<=!?Xq`$Sq z_N$YV(^-cRRwky+t>qgz!<`z0<@rZ1Uf|A-wH+}}=qd?*@+A4TkD=lB!n%FF_KQcc zi0rp--;!zmzo8WWD@THoN33;7yN8Nr6FNo zYmb^u^2O1%(d)&inM5?~i)LL`Rnb zfCVr&t|f~~{$c|7qVX}+=)=`#11+CBJE@Nx`4DqN1F^yYHd~v}4-VMb@yI@Xph=wV0flLFT83!^0IN4_Tc~a5=RDdyA zITRtOsi_GTm@ThrV8ARSjziq7?Chjmro!*-Rd0<4o5oFl$@>b40RXsNrNKH_!P(h% zv;FubZ`N&i@|dq>T5};{=KdpqmP;X=H(J23Crh2(sR3%VYC55-s|x}o?t~ekEufVO zgV+P+8@AjFM<;r17@nI5O#5^E=YRTzHArW$dWJ#yh7)22!IOg%I8RKC}{B~!o9>T-uqv0o-CT4nX!4> zI8x*Jy{h&ijvEzJaLE{==QAp$g~653f|&I==abIOqz+WMyPjR8{pV0=@MG z0AS9C0n`f6*LAw*S58*qH6^7VGIV!SfA=X1aOBy&aed34A|4F!~>uG)g;~tN*#o$ZF6xnE6gA@@dx{~vB0@= zF`1il-y7K!fC!`j+I(5fxisRrs5xi<{cUsge?$FK8W<_}CG(aSya4VX;Ysa&`u;sO zZrm>JQQZw3zSr+C)ynd+1-oV=$s+8bix$hVzuoDxG4sV1Y+U9rn(6cB&!643X=ljk zKyrH1%Hc8r)5;h7QJ-&$-EL#P;B->8(75afarH_#kq0X`nV6Us%mQRDbAGH%<>%vj zTUM6k!+S(bOw8JHwARb+&2(>8Z%`bHm65SOH;jZaudvXp#3m{)Z}#~qWwj)k;S6P_ z_DOPpxr7ZJd~mi7pV;I9yUfFvbNAM!G>IsCQWO7?9!TrGAMdB8rXYfju%7Sg@89?r z&vP(f{ttNG#*u+|?YJg^_8%|~Q2oC$@L1j6yn|2hi^DFNUhsbtdj138|DylD zmlE&xf5rPxA`>p&zjg(+Si37kbh+XS9&1()9#l;IJQygk1<`@{n6Au}6iykL>ucEa z4_4BzWxNN!N2t>gI^S}JB5)KFD;{LY`BzI6!MiF1k%sd?+%h|Hk`RNmnpkVW<$7no7-z7gKB#A?AhzU(tvBZOaPcH4}}axPenzQlam7h-2Ko$AmD6J$`&Rs0NtxJ9hHsy7+jOk-f{PWVwjeU(vmrYA!CtNlENye8Al=^Wd<;!otBj zHLB+3$rU5l&^&mfiQm7q=?TYvME8rNOyWQKoP!lvYnqzQPfb~o7a}ga{QO+!zV!Ex zpSg^xKPi6pY-Lr|$L40kIojKI?)1{0FawqBG$#iK$NBT$-j7yp@9vTwK0N+~0}@Xx zR6hp?ht@@>z|e3$v`caMO|aKF9lj`))9<2_hIIAytIEr}%uLSnAjOAlN(S271<#%P z8WdWQoQ!PC!e`6+U2on@|H9^`qw~T<=;JRxG(KK~2j}hW-T!S?XqPtPl#dVCMVb53 zrP1Nxgkxwuk39(E=+8GlF@a(mvCQ>|{fFk>#S0fqYabqE!eNmDg3Ufi0^?Jz#yL8b z6%~dKriZb}Z4Lne0Un;K;#ofz7ujiO2(4HQ;&)X=Wo>N@*L&mLn>U)`QO6+nL_dE{ zr?0QCyvu1mXz_<4Dk|y+og50!qsFrz929irft!FCnJ@-aGIee3$gr@m@^`ZgSb01i z@xB7({y4C5dG_NjZlFuZAHW8?PV@6C-z-Eh$1bX=sYQREg+$QicP+!o(J`Xqn`cHw z2IIHQqr}%F_GyadPjYi>2iG6-*@bN3_F;6?EYAMM1jS|4n2#4qX_e}NE-L#3;UOmv zz7ogCSXo(lt)kVG_M1*=v49K_S%#E;0i5 zFf%hd`B}89jt0k3kt?kAtHk^dg+;g%4}Pz&udk~nl*zoZC?;*ng&9jD_Uzd+0PV4{ zu{_x8FRq zseSmpqvJ^7;S^*U3g_6wguehsqmrsBWLkF#GRzZH?sjmkFo!k`4(HEKdl_X(aE%IZ z5|K?i6zEk|Rx&a&rWy>e zWoS)LPd|lYs2>LpI*G9xETt;Y)Arj311X>6;4t4>`a!`Y6cQTR&q^C|jEpZT$cEHC ze)xJaoI*WatcV5r3mVkUOtl0LP&l%)d3=0);%{dw5UP_m=XxxtBYeD85%X zF@7Bj6-o=~2}~v&36OrES6~!y-}|=jcv36V&Qx01l+jo#nkQM#+R7??1^WSUQc}`5v4>mts%h;?(8pySmV5u%+1q>QA8y4F)+<5eIZsA0 zZ$Iuoq2-#Jp9j+0o-P8yCwx2soju7;N9XkO1C{@?t*h$l>esH(EpOr=?-$l=48R^L zswuUQ5wcQQ)Y`w=ckz>`f3$D>P}zCujVzVe@rj8hJ`8Z^E8kdhkzD@qZDEgH(P#}}P$sT+$zy+dS zc7A@f&q1v`n}NAGheWPwO^{N>{l!F=4`*`n^5n`+2=b4CQfX~%9qk-;`s7Ke?TN?) zBCyrJd-wG(R^xxv^BXg|Dpb7nbG#v+Bt?}sNbAsAT3WA7Dl9K_W`i208YiApRJ63W zy_%<2An80wiR-HE`lY6-Dxtr`HQN;OoqZ3>kzmll+x_P<^#PY+D0A5klxfP??c}2) zrvl27SvnQ+E!~bwd*ImEhL;<{^67gm%r^oVc*cT-YdJ6iX0;}pY5xJa+7eEHzh3b! zjMT8u(Vg*Fn*s$^>y&Jc&P@D+-3|MJEduHg>`76VGS{U@a4Oyh82hc(==H@>pHJbO zEa&ddjnrIZB0iE6ct|xm+_eW){PIsjA!4I%&}XdzT^50*7XCt$8V_CPeSdQL$+p*(kJ!vKZF;LdgQsx=p`(V=#GvO7)D z)YR14&qRvlP0+8d1}wf`DUkEk9}70@){j8;{gs$HjR_DEcfmsh`NpaiYtzVz%gVkE zJ*jUi8}*JK6My{ZQ2O;wdj_cl)l6!Q+5mCenp%!W5NK2CjC%gEoB?E3mO zW`BKoc^k+Bu+Qfa5h)In0H>j^Q+ao|GMGs?QQ9-w(_}XOphsC4aJ=w|6XRVea;2rE zaf=%v2J(-vMgIBpK^7k2to``&v*P07E}*pazeYypsQEPzHu0Ea=Ut|{eznFpzZd!k z({F>1Lo+v>+=*m&>R(~0s<(|h(fx#!l*y=y$2FqPf2Ow$6lBl|{X9KZJ1#j-wuN)b zS>1+shQ8XjJbmGXnPYE)NEpn?%`RwDYV&K2HPbuXhW3ep@@Xz2I zahFa>f0?D1*B@9%2}$S0Rk-!7TesHqZ7Hf(t%v_A|A$8sj=jO+%sH_@?BjvgX{$?1 zxG#-ChskL*M$lM6_o^wzK>zQ}*J~&c*rRfJ)fz7$F501HUC+ZqvV<|GU(e7mhh-zK zS1*PfT97;rl&5;8lC?E&d)TvQ3>pN9+mG;|?{6ic?dAmrAtloFcJd4jO7+dr~1#|^G%7fwF-j*YFcx%hu z)tQl8)!E+j`v1HPy9JGN-nmitpU>#I)!aQide%UwKD8&=HFM@NrEYyshBB9L)6(W< zOep%adSct|v-(u|;7SR48XA#19jrp%?RM6ti?PNE&I53P1_hE0Jwfm!Oljo>mR~Es zTG1C|S@mB0-6Wz9(v$}B`V4WwCb#}!Izn{Nz#yExezM(LCuB-nKJMk~ThDRL`-9mVy|ozgLpk&X62g3%EX%X9BKdoJ5} z1my|>HJ=cU>)fH}l^vaud%p{OKE-ctwC>UuZD>ZB`TTK}T!H-Kz1++c7u?-HH#cXs zSX@t3Gbw_~R*w!liCD8s072|}FaNytmy2T0v*R>0c%iNkQaN05;U&glT5l{CYhq&Z z=?oArZnalk1E&Q?Asw7jj%Jl|50|Y0OdJppF!UPex7?36|28#mcP3o?D3khyH$0U6 zlG`csxqud1*!s>Y5GyI+b%tvoSY4K z%+&1P&7v?xz8?(b5;!A2Lo^%f5GpIgoO0XV-MwDy_R*91@nY*ejD~4KbbdjBE=$*T zh~9;lm0-VQwry~jDQiUrxqg&Vu*)|xc?CXVqk@5$KBMYZ!Q?%?A-k?F!Pu^^f44N( z|2xq?ZZJPPJK6Q0R)vJW&3<)YN*;1H(rDUcLMQL*tImHPNi5S++3OLt9e0Xb8v4BY zUYeqI-l|CwHdLijc70#thOKqdE7>Af7K$4mbBk)Bp!Lp;xhK5D>>}Kwa(3W;Ffoka zG?M^TcDLo{3v9I7x7%x7Oa&Fq<+QxK{9OAuDf>f?d;%%;@$l!+S7>yo`xCtZ0MdL| zv;YiS??ui}I5y)V`Rms&pf9*rV7E%+uCIS>06z4NA8*$Glvl)hoSbDa^zPXo&2497 zBO1Ga&Qw%C$-buOFfHCN`ne55+Yq(0m!X{ZHHB^IN<8WvJ>- zRb3__yoA(HUSbcyj;Sx`_KyP0q6Krga3*rM&l%hIqcPAqCff435lt~-{~P9beqUEQ z*7Kjmtn%SGX0O62Z54?rjD%jS^-&_XL)DdgfU zT^UjC)QwNPrmq_fNO)545Nc_kfvvBAt@Tl8LvHl+@wk`gmi_$_=|L<@b#`=dxj9~% zO&heN+z|S~=>Wb zdqeJ>9%*+`_YbP~T5un0tOb=FFSwuoI<~H9zfe`Y_J?4VD8sgqA_t zVDhJqq?uuEo%`i~`!t(I%(=*j(7Ei(%@S%?=b`(X7Q}cTT`hfYs-rK4tf5MAIJPpD z8LilyhvX&#gRmWnO_cG9(*uRu7pT&1$rJ)zC}eg1UC)GxYk6 znzM_(a`!W9uaJ#M&vf*)H_@EuyN<~>t%{E3{OP2s2-8JWbh2R z3;U;F$L5{n%!-yLZ;-W3%nSP6533d{seNo;fEF^XY;v7kUGzDKk2Amqps^wnAbh9u zr7SF#;H#YX_qGp_9+|45dCW&BYo+pP_z)k)DH2&$MmZwA(qWf|exek4T65Lk$jWnL zZo0pKgDF6o2WjF$~6xU4`3A9+S;?YBKyI!!}il|6)D2s+*d_OZm(=ar}X!J z4qRv}j(-AFSr@B5hwHuH(wtr82>L0nMtt&;FFOe$qWUePFzHCVJnpn%n5ky zuwHZvxhcn)e$=jdo4H!-0X?VtX7-{;F^=P&4%!SWWKcwq5G^k}d9oRzb#`XPTZ}(F zh1d|AYCl2`!e&2P>f;u{6DJm4K zx$o~TZBNkWG#@}FgSh$n?Hf=3ZoFDMz*hk?wp3tqpzrj(NDVxELXmax z020g&2>(xa*WE!B1T`l?;88Uj4nI`=KulD08DJ13?I$O=iu}b5JXW~5xfQQmdAR;X zf4sKf=lJ-|GqL5f=h+I7g|3(DHsMn>0`&GhshLnBNe6#Dk(&Yc^%%Yv`Ex4m);9_V!` zgj9ndU#yv>o)mgtCzEh3D)9isARa8FAo3$eHda>~&*{zwS61{|q$9GZeqR%z&CSg~ z%mA%}!y=4{;dpW8Y*H|kde`t~LCKBgpceXNYK#FrNNu2cZOt1Eaz>uOkYS6i&CJXI zm@@Y*#b+qqatDqKgMj8yMcx8f3JOh}LU>ZrA}DG&7@Df&<>Na+!4dW4i4(E&S4UjT z%x9QEl=ZZ=RmUA*Fc^3Ve-c__j2h2+{kr4J7m!gHPa+Z$o(Zd_ zAfJ6l9GE9A6Aks;dq=4vhBJ3N?JNoA>M>B*W@h->B}JMql^oF1(>t=@QO^}pG_DpE zibkWa{tjho-6xzXqK;^_tfLT@-Ljfg@f|{p_u|ro5*=9Y!mwsVVr%q3XwBe$o)n@7UGZ**QgJb8EM#gCbpI z1a$U%h77HoFF%tZwjAwISa&1+RSKVj+u>-~U@$n7`-2CLDasbF;^J(Y+=!EirVDp` z0259@{2~1)>AXW|^m&YcK~epaXTcQDnl>? z8nWMtEF7wr6`Y*jl2wWYq6t%tF!iTRP1XJw>Au+t+{0E z>RJ*QNXqmY1P{CSkWiQQrPwcdOF{qA)6?_r#l}GDtp27T27VEu+vWc>CCH|1^(O5% zH7}(rMN%GICk)_VO!F515tux1p=jFT zG}_$!3p!#~NuVd{8W|rCHSijvxxH^oOA8Ruot+(^Zi$IYeG0_!co6pX*9(2#93v6j z2C|vNU9y9DdgImV}e@r1xxtDGwiQYJzH$$PVt1JJ7!~jyk_z$S_Yj$6Q zd~zvp3=GX5Kf=8K0gNu)dWP;m7fbz-?}k}^N{!v5nS(fI2=7dgoD~iQZNu0S{+&EW zOsl2~_?!o{1#>J5zwc*pA23T7v13GUcP83jZA1~LA-=W;__MLGF%gAJP>Sq6 z!_q-8NI_B%^Wm+X9d9T`Lv^!X3)#6o#YxzHd#-~4lgMCUqQAc#(1(QCx@NqY0SdH_ zo60>#I%fF4Uj<0X?%yesUn`(UMhHeQl!=TGhb|2$wiIA%&ksL<0hDmbPe)+RF}-_} z#TV?L$>N*1S5Z-M0L|_4uq750@yh^uf3(7*n}jhb@I?R$l{j(oKQO0>?-6{&=t(8} thaG0!70Va#5OAP}{QFf21kr0dKwVy35LrLPqzbPhO7dz~Udx(4`X3eV#)SX? From 61ecf539731b7b283539ea9f65883dac76ff9ac5 Mon Sep 17 00:00:00 2001 From: Chris Doherty Date: Tue, 28 Feb 2023 09:07:40 -0600 Subject: [PATCH 08/14] Add LastUpdated filelds and update MAC validation Signed-off-by: Chris Doherty --- design/20230222_tinkerbell_crd_refactor.md | 54 +++++++++++++--------- 1 file changed, 31 insertions(+), 23 deletions(-) diff --git a/design/20230222_tinkerbell_crd_refactor.md b/design/20230222_tinkerbell_crd_refactor.md index f205b30..21eaf9d 100644 --- a/design/20230222_tinkerbell_crd_refactor.md +++ b/design/20230222_tinkerbell_crd_refactor.md @@ -73,6 +73,7 @@ Users resort to Q&A in the Tinkerbell Slack to determine what fields are require - To change the existing relationship between Tink and Rufio. - To introduce additional object status data typically found on the `.Status` field. - To support new technologies such as IPv6. +- To define new Kubernetes events. ## Proposal @@ -83,6 +84,8 @@ The CRDs will be developed under a `v1alpha2` API version. #### `Hardware` ```go +// +kubebuilder:printcolumn:name="Cluster",type="string",JSONPath=".spec.clusterName",description="Cluster" + // Hardware is a logical representation of a machine that can execute Workflows. type Hardware struct { metav1.TypeMeta `json:",inline"` @@ -212,8 +215,8 @@ type Instance struct { // NetworkInterfaces maps a MAC address to a NetworkInterface. type NetworkInterfaces map[MAC]NetworkInterface -// MAC is a Media Access Control address. -// +kubebuilder:validation:Pattern="^([0-9A-Fa-f]{2}[:-]){5}([0-9A-Fa-f]{2})$" +// MAC is a Media Access Control address. MACs must use lower case letters. +// +kubebuilder:validation:Pattern="^([0-9a-f]{2}:){5}([0-9a-f]{2})$" type MAC string // Nameserver is an IP or hostname. @@ -375,6 +378,9 @@ type WorkflowStatus struct { // started. StartedAt *metav1.Time `json:"startedAt,omitempty"` + // LastUpdated is the observed time when State transitioned last. + LastUpdated *metav1.Time `json:"lastUpdated,omitempty"` + // State describes the current state of the workflow. For the workflow to enter the // WorkflowStateSucceeded state all actions must be in ActionStateSucceeded. The Workflow will // enter a WorkflowStateFailed if 1 or more Actions fails. @@ -390,19 +396,21 @@ type ActionStatus struct { Rendered Action `json:"rendered,omitempty"` // StartedAt is the time the action was requested. Nil indicates the Action has not started. - StartedAt metav1.Time `json:"startedAt,omitempty"` + StartedAt *metav1.Time `json:"startedAt,omitempty"` + + // LastUpdated is the observed time when State transitioned last. + LastUpdated *metav1.Time `json:"lastUpdated,omitempty"` // State describes the current state of the action. State ActionState `json:"state,omitempty"` - // Reason is a short CamelCase word or phrase describing why the Action entered - // ActionStateFailed. It is not relevant when the State field is not ActionStateFailed. - Reason string `json:"reason,omitempty"` + // FailureReason is a short CamelCase word or phrase describing why the Action entered + // ActionStateFailed. + FailureReason string `json:"failureReason,omitempty"` - // Message is a free-form user friendly message describing why the Action entered the - // ActionStateFailed state. Typically, this is an elaboration on the Reason. It is not - // relevant when the State field is not ActionStateFailed. - Message string `json:"message,omitempty"` + // FailureMessage is a free-form user friendly message describing why the Action entered the + // ActionStateFailed state. Typically, this is an elaboration on the Reason. + FailureMessage string `json:"failureMessage,omitempty"` } // State describes the point in time state of a Workflow. @@ -410,17 +418,17 @@ type WorkflowState string const ( // WorkflowStatePending indicates the workflow is in a pending state. - WorkflowStatePending WorkflowState = "Pending" + WorkflowStatePending WorkflowState = "Pending" // WorkflowStateRunning indicates the first Action has been requested and the Workflow is in // progress. - WorkflowStateRunning WorkflowState = "Running" + WorkflowStateRunning WorkflowState = "Running" // WorkflowStateSucceeded indicates all Workflow actions have successfully completed. WorkflowStateSucceeded WorkflowState = "Succeeded" // WorkflowStateFailed indicates an Action entered a failure state. - WorkflowStateFailed WorkflowState = "Failed" + WorkflowStateFailed WorkflowState = "Failed" ) // ActionState describes a point in time state of an Action. @@ -428,30 +436,30 @@ type ActionState string const ( // ActionStatePending indicates an Action is awaiting execution. - ActionStatePending ActionState = "Pending" + ActionStatePending ActionState = "Pending" // ActionStateRunning indicates an Action has begun execution. - ActionStateRunning ActionState = "Running" + ActionStateRunning ActionState = "Running" // ActionStateSucceeded indicates an Action completed execution successfully. - ActionStateSucceeded ActionState = "Succeeded" + ActionStateSucceeded ActionState = "Succeeded" // ActionStatFailed indicates an Action failed to execute. Users may inspect the associated // Workflow resource to gain deeper insights into why the action failed. - ActionStateFailed ActionState = "Failed" + ActionStateFailed ActionState = "Failed" ) ``` -A `Started` condition will be used to indicate the workflow has started and is observed based on the presence of `Workflow.WorkflowStatus.StartedAt`. It's severity will be `ConditionSeverityInfo` and its default Status will be `ConditionStatusFalse`. +A `Started` condition will be used to indicate the workflow has started and is observed based on the presence of `Workflow.WorkflowStatus.StartedAt`. It's severity will be `ConditionSeverityInfo` and its default status will be `ConditionStatusFalse`. -A `Succeeded` condition will be used to indicate the workflow succeeded indicated by a status of `ConditionStatusTrue`. When an action fails, `Succeeded` will become `ConditionStatusFalse` with severity `ConditionSeverityError` and the `Reason` and `Message` will be propagated from the failed `ActionStatus`. +A `Succeeded` condition will be used to indicate the workflow succeeded. It will default to `ConditionStatusUnknown`. It will become `ConditionStatusTrue` with `ConditionSeverityInfo` when `WorkflowStatus.State == WorkflowStateSucceeded`. When an action fails, it will become `ConditionStatusFalse` with severity `ConditionSeverityError`. Its `Reason` and `Message` will be propagated from the failed `ActionStatus.FailureReason` and `ActionStatus.FailureMessage`. #### Conditions The conditions data structure will only be available on the `Workflow` CRD but is designed to be applicable to other resources. It provides the foundational components for extensible observations about any resource that it resides on. ```go -// ConditionType +// ConditionType identifies the type of condition. type ConditionType string // ConditionSeverity expresses the severity of a Condition Type failing. @@ -461,8 +469,8 @@ const ( // ConditionSeverityError indicates the condition should be treated as an error. ConditionSeverityError ConditionSeverity = "Error" - // ConditionSeverityWarning indicates the condition shouldbe investigated but the system may - // continue to work. + // ConditionSeverityWarning indicates the condition should be investigated but that everything + // may continue to function. ConditionSeverityWarning ConditionSeverity = "Warning" // ConditionSeverityInfo indicates the condition is informational only. @@ -504,7 +512,7 @@ type Condition struct { // LastTransitionTime is the last time the condition transitioned from one status to another. LastTransitionTime metav1.Time `json:"lastTransitionTime"` - // Reason for the conditions last transition. + // Reason is a short CamelCase description for the conditions last transition. // +optional Reason string `json:"reason,omitempty"` From 68cff61d87c686de3c82cf87bdf7e255ad10004e Mon Sep 17 00:00:00 2001 From: Chris Doherty Date: Tue, 28 Feb 2023 09:15:32 -0600 Subject: [PATCH 09/14] Expand condition descriptions Signed-off-by: Chris Doherty --- design/20230222_tinkerbell_crd_refactor.md | 44 +++++++++++++++------- 1 file changed, 31 insertions(+), 13 deletions(-) diff --git a/design/20230222_tinkerbell_crd_refactor.md b/design/20230222_tinkerbell_crd_refactor.md index 21eaf9d..d25bc4e 100644 --- a/design/20230222_tinkerbell_crd_refactor.md +++ b/design/20230222_tinkerbell_crd_refactor.md @@ -339,16 +339,6 @@ type Volume string #### `Workflow` ```go -// Workflow describes a set of actions to be run on a specific Hardware. Workflows execute -// once and should be considered ephemeral. -type Workflow struct { - metav1.TypeMeta `json:",inline"` - metav1.ObjectMeta `json:"metadata,omitempty"` - - Spec WorkflowSpec `json:"spec,omitempty"` - Status WorkflowStatus `json:"status,omitempty"` -} - type WorkflowSpec struct { // HardwareRef is a reference to a Hardware resource this workflow will execute on. // If no namespace is specified the Workflow's namespace is assumed. @@ -448,11 +438,39 @@ const ( // Workflow resource to gain deeper insights into why the action failed. ActionStateFailed ActionState = "Failed" ) + +// +kubebuilder:subresource:status + +// Workflow describes a set of actions to be run on a specific Hardware. Workflows execute +// once and should be considered ephemeral. +type Workflow struct { + metav1.TypeMeta `json:",inline"` + metav1.ObjectMeta `json:"metadata,omitempty"` + + Spec WorkflowSpec `json:"spec,omitempty"` + Status WorkflowStatus `json:"status,omitempty"` +} + +type WorkflowList struct { + metav1.TypeMeta `json:",inline"` + metav1.ObjectMeta `json:"metadata,omitempty"` + Items []Workflow `json:"items,omitempty"` +} ``` -A `Started` condition will be used to indicate the workflow has started and is observed based on the presence of `Workflow.WorkflowStatus.StartedAt`. It's severity will be `ConditionSeverityInfo` and its default status will be `ConditionStatusFalse`. +##### Workflow `Started` condition + +A `Started` condition will be used to indicate the workflow has started. Its default status will be `ConditionStatusFalse`. + +When `Workflow.WorkflowStatus.StartedAt` is a non-nil value it transitions to `True`. + +##### Workflow `Succeeded` condition + +A `Succeeded` condition will be used to indicate the workflow succeeded. + +When `WorkflowStatus.State` transitions to `WorkflowStateSucceeded`, it will transition to a status and severity of `ConditionStatusTrue` and `ConditionSeverityInfo`. -A `Succeeded` condition will be used to indicate the workflow succeeded. It will default to `ConditionStatusUnknown`. It will become `ConditionStatusTrue` with `ConditionSeverityInfo` when `WorkflowStatus.State == WorkflowStateSucceeded`. When an action fails, it will become `ConditionStatusFalse` with severity `ConditionSeverityError`. Its `Reason` and `Message` will be propagated from the failed `ActionStatus.FailureReason` and `ActionStatus.FailureMessage`. +When `WorkflowStatus.State` transitions to `WorkflowStateFailed`, it will transition to a status and severity of `ConditionStatusFalse` and `ConditionSeverityError`. Its `Reason` and `Message` will be propagated from the failed `ActionStatus.FailureReason` and `ActionStatus.FailureMessage`. #### Conditions @@ -485,7 +503,7 @@ type ConditionStatus string const ( // ConditionStatusUnknown is the default status and indicates the condition cannot be - // evaluated to either ConditionStatusTrue or ConditionStatusFalse. + // evaluated as True or False. ConditionStatusUnknown ConditionStatus = "Unknown" // ConditionStatusTrue indicates the condition has been evaluated as true. From bb91b1ff108a99715ada088e21916cdad28600c8 Mon Sep 17 00:00:00 2001 From: Chris Doherty Date: Tue, 28 Feb 2023 09:56:11 -0600 Subject: [PATCH 10/14] Add printcolumn markers and List objects Signed-off-by: Chris Doherty --- design/20230222_tinkerbell_crd_refactor.md | 115 ++++++++++++------ .../workflow_state_machine.png | Bin 16265 -> 18521 bytes 2 files changed, 75 insertions(+), 40 deletions(-) diff --git a/design/20230222_tinkerbell_crd_refactor.md b/design/20230222_tinkerbell_crd_refactor.md index d25bc4e..9ad749e 100644 --- a/design/20230222_tinkerbell_crd_refactor.md +++ b/design/20230222_tinkerbell_crd_refactor.md @@ -84,16 +84,6 @@ The CRDs will be developed under a `v1alpha2` API version. #### `Hardware` ```go -// +kubebuilder:printcolumn:name="Cluster",type="string",JSONPath=".spec.clusterName",description="Cluster" - -// Hardware is a logical representation of a machine that can execute Workflows. -type Hardware struct { - metav1.TypeMeta `json:",inline"` - metav1.ObjectMeta `json:"metadata,omitempty"` - - HardwareSpec `json:"spec,omitempty"` -} - type HardwareSpec struct { // NetworkInterfaces defines the desired DHCP and netboot configuration for a network interface. // +kubebuilder:validation:MinItems=1 @@ -228,9 +218,37 @@ type Nameserver string type Timeserver string // StorageDevice describes a storage device path that will be present in the OSIE. -// StorageDevices must be valid linux paths. +// StorageDevices must be valid Linux paths. They should not contain partitions. +// +// Good +// /dev/sda +// /dev/nvme0n1 +// +// Bad (contains partitions) +// /dev/sda1 +// /dev/nvme0n1p1 +// +// Bad (invalid Linux path) +// \dev\sda +// // +kubebuilder:validation:Pattern="^(/[^/ ]*)+/?$" type StorageDevice string + +// +kubebuilder:printcolumn:name="BMC",type="string",JSONPath=".spec.bmcRef",description="Baseboard management computer attached to the Hardware" + +// Hardware is a logical representation of a machine that can execute Workflows. +type Hardware struct { + metav1.TypeMeta `json:",inline"` + metav1.ObjectMeta `json:"metadata,omitempty"` + + HardwareSpec `json:"spec,omitempty"` +} + +type HardwareList struct { + metav1.TypeMeta `json:",inline"` + metav1.ObjectMeta `json:"metadata,omitempty"` + Items []Hardware `json:"items"` +} ``` #### `OSIE` @@ -238,6 +256,14 @@ type StorageDevice string `OSIE` is a new CRD. It enables re-use of OSIE URLs across `Hardware` instances. ```go +type OSIESpec struct { + // KernelURL is a URL to a kernel image. + KernelURL string `json:"kernelUrl,omitempty"` + + // InitrdURL is a URL to an initrd image. + InitrdURL string `json:"initrdUrl,omitempty"` +} + // OSIE describes an Operating System Installation Environment. It is used by Tinkerbell // to provision machines and should launch the Tink Worker component. type OSIE struct { @@ -247,29 +273,16 @@ type OSIE struct { Spec OSIESpec `json:"spec,omitempty"` } -type OSIESpec struct { - // KernelURL is a URL to a kernel image. - KernelURL string `json:"kernelUrl,omitempty"` - - // InitrdURL is a URL to an initrd image. - InitrdURL string `json:"initrdUrl,omitempty"` +type OSIEList struct { + metav1.TypeMeta `json:",inline"` + metav1.ObjectMeta `json:"metadata,omitempty"` + Items []OSIESpec `json:"items"` } ``` #### `Template` ```go -// Template defines a set of actions to be run on a target machine. The template is rendered -// prior to execution where it is exposed to Hardware and user defined data. Most fields within the -// TemplateSpec may contain templates values excluding .TemplateSpec.Actions[].Name. -// See https://pkg.go.dev/text/template for more details. -type Template struct { - metav1.TypeMeta `json:",inline"` - metav1.ObjectMeta `json:"metadata,omitempty"` - - Spec TemplateSpec `json:"spec,omitempty"` -} - type TemplateSpec struct { // Actions defines the set of actions to be run on a target machine. Actions are run sequentially // in the order they are specified. At least 1 action must be specified. Names of actions @@ -290,11 +303,11 @@ type TemplateSpec struct { // Action defines an individual action to be run on a target machine. type Action struct { - // Name is a unique name for the action. It cannot be a templated value. - Name string `json:"name,omitempty"` + // Name is a name for the action. + Name string `json:"name"` // Image is an OCI image. - Image string `json:"image,omitempty"` + Image string `json:"image"` // Cmd defines the command to use when launching the image. // +optional @@ -334,6 +347,22 @@ type Action struct { // See https://docs.docker.com/storage/volumes/ for additional details type Volume string +// Template defines a set of actions to be run on a target machine. The template is rendered +// prior to execution where it is exposed to Hardware and user defined data. Most fields within the +// TemplateSpec may contain templates values excluding .TemplateSpec.Actions[].Name. +// See https://pkg.go.dev/text/template for more details. +type Template struct { + metav1.TypeMeta `json:",inline"` + metav1.ObjectMeta `json:"metadata,omitempty"` + + Spec TemplateSpec `json:"spec,omitempty"` +} + +type TemplateList struct { + metav1.TypeMeta `json:",inline"` + metav1.ObjectMeta `json:"metadata,omitempty"` + Items []Template `json:"items"` +} ``` #### `Workflow` @@ -362,14 +391,14 @@ type WorkflowSpec struct { type WorkflowStatus struct { // Actions is a list of action states. - Actions []ActionStatus `json:"actions,omitempty"` + Actions []ActionStatus `json:"actions"` // StartedAt is the time the first action was requested. Nil indicates the Workflow has not // started. StartedAt *metav1.Time `json:"startedAt,omitempty"` - // LastUpdated is the observed time when State transitioned last. - LastUpdated *metav1.Time `json:"lastUpdated,omitempty"` + // LastTransition is the observed time when State transitioned last. + LastTransition *metav1.Time `json:"lastTransitioned,omitempty"` // State describes the current state of the workflow. For the workflow to enter the // WorkflowStateSucceeded state all actions must be in ActionStateSucceeded. The Workflow will @@ -377,7 +406,7 @@ type WorkflowStatus struct { State WorkflowState `json:"state,omitempty"` // Conditions details a set of observations about the Workflow. - Conditions Conditions + Conditions Conditions `json:"conditions"` } // ActionStatus describes status information about an action. @@ -385,11 +414,14 @@ type ActionStatus struct { // Rendered is the rendered action. Rendered Action `json:"rendered,omitempty"` + // ID uniquely identifies the action status. + ID string `json:"id"` + // StartedAt is the time the action was requested. Nil indicates the Action has not started. StartedAt *metav1.Time `json:"startedAt,omitempty"` - // LastUpdated is the observed time when State transitioned last. - LastUpdated *metav1.Time `json:"lastUpdated,omitempty"` + // LastTransition is the observed time when State transitioned last. + LastTransition *metav1.Time `json:"lastTransitioned,omitempty"` // State describes the current state of the action. State ActionState `json:"state,omitempty"` @@ -440,6 +472,9 @@ const ( ) // +kubebuilder:subresource:status +// +kubebuilder:printcolumn:name="State",type="string",JSONPath=".status.state",description="State of the workflow such as Pending,Running etc" +// +kubebuilder:printcolumn:name="Hardware",type="string",JSONPath=".spec.hardwareRef",description="Hardware object that runs the workflow" +// +kubebuilder:printcolumn:name="Template",type="string",JSONPath=".spec.templateRef",description="Template to run on the associated Hardware" // Workflow describes a set of actions to be run on a specific Hardware. Workflows execute // once and should be considered ephemeral. @@ -454,7 +489,7 @@ type Workflow struct { type WorkflowList struct { metav1.TypeMeta `json:",inline"` metav1.ObjectMeta `json:"metadata,omitempty"` - Items []Workflow `json:"items,omitempty"` + Items []Workflow `json:"items,omitempty"` } ``` @@ -527,8 +562,8 @@ type Condition struct { // +optional Severity ConditionSeverity `json:"severity,omitempty"` - // LastTransitionTime is the last time the condition transitioned from one status to another. - LastTransitionTime metav1.Time `json:"lastTransitionTime"` + // LastTransition is the last time the condition transitioned from one status to another. + LastTransition *metav1.Time `json:"lastTransitionTime"` // Reason is a short CamelCase description for the conditions last transition. // +optional diff --git a/design/images/tinkerbell_crd_refactor/workflow_state_machine.png b/design/images/tinkerbell_crd_refactor/workflow_state_machine.png index e85b4c3f0a83815b2508fe402ba3f9008d8233f0..1ef909e94e98c5bbb6717f44093dc9767985a266 100644 GIT binary patch literal 18521 zcmdVCc|4Wv+b)hm$XLsic?gS$O6GZ1#&t``JVqIlAynoiV<=@F!c9aXnHtPwC^8gE z=9F2HVV|p>_q*SB@9(?!{^$3{?~mu>>0ax;?(4pW^E}SuIIb0Kpr=Vq$x2B?L`1Ev zrDjA#L>veItsNqPE6+!-oQ6L_KI*1EcJ3bjZVrwoVh@lz3^w>mmeG+lH@QH_uA@?;;UL+|o zd`e3;mCNm>V)2svOulcOt6fctFXfkskJFFnwQekSpLLo^uRdWET9*9e-OQ7GW?M_u7l3?S)(4H4AM2DQ9Cqqpysa5Kukw|wv5|6#RHZXkk;;)`ekENdIyiYA+sgdt zuCkx51`ZR)7)*a5zhsx>>1IRnjRS#mOF|;;JxfGH8lKu}$~eE#?-~9Un8E#h@2;R; zY7gz?qx9Uo&pdQ>bz68inOW5xw9OLn$DZ3<)xKJEKbr@yg}-f{|A->`kuCRa-Cr+N zTKEkOw{HkN4lXaB{kSvZz4XcE`Rg%1`Tfu0UA0b*Jq@!RzvX{^Iu*aGdWf5`*UpVZ z4jLsLT6}fv#j_L9WtZw49UUucp?Q%ZW0JFH z&n7$N(WChL4{}r^t4~l1?tgxDp!yixG9MKm?`|GoYim2~1D$we_xa^5jUp79c9Vf; z9r1(*SGV3CswAdB)XyoasNC*HU%QUnBz)HzKGP_nEO$7-fZ`1h!aGoKL7UHBeq4Zm zd>|$lhIcY0W+Tyo*J59eJOfuvUT&?;*7<%%`;A3KMR#f?;0FKpo}Ld<6y4EB1coi( zJC3U79v4m)7aJFPU6zMl=xUQm+la##&-B;(drOk!pzvZvSQsq$5Jv4ko{sY1F!0&` z@-g)6Pv>Bm|9lKX{^xFl{?Fb2^W*<{>puoKCuX_ik2J`SWk(O?xsv zbA4alMDEXq>}eL*gFif?;RBRH(;Z%wcXc zxYnzv=<<#a#V2@OzyvT&vR6!hjaSsz_w@EYQW6{-%&T#y&oU>ck@Ba_h3J+}x~GCl z`!8^Mp(szj4?5ZyB)i>R>plDAu=cadgQ!jl_WIJey4d9pr6Up4$DZuIE6^=(*fe_+ zS--da?LwIa{h7lo5`27o%|k;qzTZb{Yio_H-uCt`eR%wFapY-*-1Q%ywl}^h?*00z zr>AFQVU1smVrcZoICufNkhb9un|{iE}? zwz-1dTdNM7oLR@uKfsk+vKJa!Sd<1EK6QNi;G_+SXp`U0Ze>v7_zi=y-(}H1I9&1dl`2jYh4DT>c6hk@%Eza@3c@{V6%6e1 z+FeDO`=_n>+2|cMzkj^``6cm%3m0H+jfyDccGutk639wV|Mc9`?jThZsfw!V?AwQt zW#J8qJIe=TFh`G5mYY4JED_L4r_1n$c;Mw)zrXi8==Y|&V%U@X@5IE!!?{_4EU2`o zC`}@}+_T`_fTLGOtL8gXgmbu?d5!MymY;l3LBbBQ zz%=;OsaJ7-=SY|HK{gzEv5-L(HIk!trlNBFP-;V>6_tpp(B7Dg7I>tWvAs}IZu{&) zxn-2lGL!ieR!+`%y=$ve+J?-#EG6bOR|lJ9LlM4!VHi-ety_nSI}P3@9K}6zdH)Ck zT~E67``+H(t}d$Z#2eA3tqCG(;a5IZe_t1)prEkjwH4heIc+(R#IGBM9u*eP$F{tu z>rpAyLFjGs(Z;=PSD$0{XXnyPscC4+EE{B!vXhgKmOgs)$hQ2s3+ac)=8=eawz3t% z8T^Zz+VBk8PAJL%dw?}bMualSs%tI} zQfjK1L^W5#?u0N{V2qjT&(qx9-e?J$mc9N%iJFJyX!cX?82ana>wM;KYiYVIeX3o9 zB`$)wYtkRvFL3Pa)f;;|{-rzvuS?criU)3Nue-RVowT`ay0N_MJX%#8yjd7R!z5<8 zI97Mo(!AKH00PAiSkqaNU)v2srSKUU>=t<|xv8R;%t}m7RFs-V*FU*&apm;n!jBa`duNy zaOB9x8uy7EtMv4A1_p*<5$I}a%kJ-=V`F3LI8vGGKkl4xV}uUI@bd7ee6nhrSYGy! zgwz?uZ)9pZH#c_(fn7w_o(>pEnrH*SAAwM_lkMXBrN< zb`36Ein{m8WDE5%$?l+bHlvl7zZJY^x>JNNWGVzb7`ug5szIMtSYl&hy1KhX_%IKt zm_+&b`NPA)M64Rqc_oXmx?g^b)g2MgJEy1T$iy%6d7n|?SX(@&)7c!fJ8OlMl=M>L zjkK{OW*$jNNh}us1%o*B_)%uQ2-v^|@)`=9Idg5K(qT~om*TJF*6W9Nw(rf9kB*LB zJcCPN`H{D_cT@-y!z4DN7K%$LPLsUyAl&C>I2BV*+9^%Xp_++c*~|KQpTeyk~ph z?d!c~<4kuaW5ig`T*i%FhAp?d__TMyJ@PyjL8rccJ;KNs!C_lu>TfA zv8-pH5rm`R1$;Y&^R{Z0`@|k0n3p^#xlq2>A&?a{&1C79qZ!Zfs0uS4rLrfJgi5gV z_xC&Xq>E499fg2O!!8pwy|c3e56OO$CFgJC#OQ5@Gz~g4bRHrT#u#F*3rRwBj+ESo`0{e zc%}edu7J_Bq0L3lHGiu0j{Tjlmnp}RNON04gqMT9BKntpj6E8>yB&3NYP#U2E`7yN zLg*v?thmndOLdQOu^ZbP+UL%xWdL}6KXJysJt2-=c44U8`%r9)2YN_)x`&#A!cfR+ zQeFGlG>S#29bQ8QU1oerZiu9op&7E*bBC^sR#9}Fa_oliA>@?~@yKqYFGqFw+RiUe zk^Rk&?ze8;+T7fXSKRZvCwgg$xaFgJ(+Mw`MfS5-)8j&KhlcjQq1S%<@a$5wK&DbS zghRn><8Fi9_6SHc*wmr2OEE+qu+eNc-q0!@K6L2U-tQ&D*}A31l5*>*>LO|}7KIy9 z)oN;L>q`?&mJInZ<1Z3(YYfYEJOber5Rf!I_6= zT*>rvXuH_gr|((i>PALJ+DY^Clde#iz-YXHB+ZVnpqyeZ6O9dmr=0t^>TdvqFm3c- zJDwkkK3SZZiTEzow6h8*F8|wTjeAx}iE9kw36nC5T0p68D?fkke-5REP0-NLV5lG~ zYhJ;;D(h$S>NaV6i}iPY-PE}fN(X}jWm*MtWo3F9GRs51mzorP<_wY*JwD&`spxwj zxYQ8oyS%*oQdApRl{75=5_@)B35`>A|Ma~7{d;4}P?HFHc&__fIDK{}f# zy&7C`2*d&v;V3mgN8u-%TUykof&|w9X+i=wn62_kJVS+NU-7WaXGh0cwO04i>;og#1Wo~6;l zh#%V2(9{%0DcfT2++hQxt1H&PcUyMXw(j~;t7vml1_#N{XIIC}j4(&n28{K&W1xyu zz?E5Aj*CDZ&=61T;E8)9aoM2$aakU=g|+!a3&cL`2PIf=k|pS`Uw${%gMxwpUWO^q zn%f}uY?J2dp5M%-JV~x!|7r5rthlIie4Io4Q043Qh5Gl3U^^!gH3B?qA4{0z#w%-2 zoiDnvxi})joYQ#E_cEdh9Vazjbx35G_LSx!6RH=#TO|D#cb!3MTCA?CQjGWF@?samf?oDgi~g zST|{}DXoTp4M=PTO)t=aZ?uOFIrnAX3JpDwq+H6g?{)2(`c_u(w)Vc+=Q9U=l0x&D zoz`aF`gOpj%eU8gQOqY6@rm-hxb;h-mRFS9sF9Gn*6FRSt>xlhlO*Z#fs9P%Sj2{< z_(e@k&2YG2(og6WWe(Q!E-B~FzCMGB(Q+%rjPk*R)6uu6goK19tE+y!Gf+o(6jf># z#~bSU?s6l=Lrt({#KTlRo#8y?csP|?q(k{@TfDxmuByp%zZKiX2WMPUhtGJ=oGU%f z$M=$EBIqZ3>V*F+`@r!s8k+~f;-zuS;<->neQk}E4OpLpx};!P(-6T_Coc+n0kW5S3!GV&bNh-Kyfp&vacWcIriL$9MVKwr7qX*I-5CO6dYpTCX_3GX15SAND4awtSn;Vo|uf{`Pl>74O8zY+xjF zn5%ImCg&-Cc%XuhCVV0~yUO*Vj!Q|ImO_570c<{ORC9G~y6fIhk)f(l$_|1(!?he% zDK96-rBm+;h#@#R3zj7=Ww1zx7m7vYHqoYu5ta*z5+Ay|&rLnVf8|TG(Oy-V;#Uz! zZoV}&J>AkQRetBrVb3KCs)7^ou4<#^1_sM>vDJ=mSv~jT&|KJ&wimgI*d*PUkDh`7 zZH+y~8w2BT97f9!X|9aI<5xE~eLX$>XVRP!)-n}C_6#rQ!CeIU0*8Tb0nP9i>k+OI z=me~t2Gr;G=<=3-{&a$z$c)fyAph_S(B>axNT3-A_l5oGfePVK42Z+F)A1v4wbB6@ zX+3qf>tISG=HZ@nE8#Idd05vFN3Vriq(ygqV{T5{i%dQICBJmI{-A(Ig z&Do2xQ|%qa-Zeght4$rf6FB-bn3`2A0Ujfwg?!lSw%>Z@EL8eyYD#6~4Spjl*CL`< z{6o1#fz359CowTVIG4XLQ0jvZ?7xN5|^j{t&zu zi$a@elUGVaO6ZZB7bV@t?=ZEtwJ}`|7OdJkmcLKvQe~Vw(PQWnLjksQZOXnj-e)zh zGAOR1-fJTwcO$%$}kdFhLFD94I)>uTrV zF_Ru3CJY!JZ*R?J>C!g!{W5UgJsSmYRC*IK(!z~Bx0$pKenu02uT@@D=ePHr>68+y z=QQHt;_Y-Tybm@?-ub!2=b{h|jbPOqe9QuXk(3rR8NpV9E9VSUv`mdGARhdA(vTGl`J{Hl?mpR=KG>S81V z)v*At${l;3%v_6Vn2BPG>DSX}z(R4M0e9!;EFBArbX&ws^!!RVD}CPg;ZW#^&3fL; z6KiQ51?tls9bSYRu{t;6D)gq!MA@d*i;9d=T6IrNlDLn(AVn&c43OjeFcQ+zX-&?; zYgseNU%c;}h;4~sdK0=ROF+9)o|CZO=I=@5yJgwA83M;bzCS+fT7*620k58&&ELg}fEyBzDb*X9JF+LlOntu2HGh#1Fh=7Xza(87YqlhUTpBq1YRrWW=Y96 znAw)ru6ts?fBib`^6ucr6?Agac2{R=)&t1hAcL4(- zW?6qo42lnO(!PD5A$B^2%h?ozcYrIDA%6h0IjlNgU&$9i6OBO?)9`LruU>`X$_yy8 zv{O#Li^D6xd++Mxd%v{2e*L=7SC}aucM%}+Hh`XGa6nBS@$)^eUTE z4R7Rh_7`JS_H>brg~7lDZs5G?b5SxHHbDnbD8x>ZZd(?6ot3g(7$}Ysdvara&TzdI zcH)MGwoD|DO2pHNEYaVg2zb2!yto?6L5YyPErC=5RrgZO5P7#1qxvQ@VHGm;lk~(y zVoz%ur~$Tx?&kuT1tVJ^F9YNV1x0oon(xp*kJ_pCSgWC6bp!p)3b{S2~oUQSzj+TD(D2B z3>M=H2ZUZ`80M(Rsebr4_EZUdNVw5 z`S-?-qj8{-lD~WVd^2+E@exQWnbqEA!|1e>&h=0P9vygzT(DG9SqCu9<_iJRq%| zqI0_XPG|pPQh-UfP`6{QwH4<9DWNKNGcy(JS`Swr`1e8!FjW7GM)UtH;r!De(0II1 z3>q4y^jH!!GY-f#Ky()tp;c8??d@8U^uT0ob@cW|X6Da2=%D!c_%<(KtsX`JYsND- zT)|tKoSrUHjD-T3iiU>fkFciuWw^rbEhmZj7cn^pC#OhKzMFrpQ%$xoS-FAA(v^C` zMsTG@KTF{cb=JirE?xnI^4tBE{(f<8?ptpOUAg$FDP&@B5HUhv!kT$UDWE3~^z`)o zWtU|AC_~=7c{9RAAknmJY`)ypSD-+kYGeI}XXA%Jj#5%m+8PsBKT&^we*@&|RndpH zq1U<6e#>0PZW$mS+N?r$Q!+9dAcjLGjf{xs;g);_K@mbj4J;?%)s|KDkc($wx-@Y( zoR*fBg~ca`f0WU0SD*rha@!MP4lL0r@=UZ+2b2f}|J5ldnWA@Vc@omo0eVb$$UjZ~6DM&d`{5;I4=4mrw zGtodU$^dS^SZPmDC%j~SWuh^IKmc0s@bbdLZ8|M0Jife^_MQ>^u??(^CPZkz<Opeyr?LhJqmH3rKTzX`gM0z;AP{bTjtA^lb7! zM4N-<_nEg3rR}rON`vKj9Or|+|2LA2a`;0}Pnqjb8SJWa=g+ePT?N6UMfBj+$&!F; zry1WA%ScJR{0(wik#QkS`nC1BelH^ewW6=Pp10=N_utC}0N`{=j1WyLA zmbNS_QBmjVR;@7|IAG!Fi95Pqp(vf-rOKiOX+vBWO9}PQaeg`a}ROFgTE1n6bXQ+t&snM3^6F+=(bU zZ-p30)4Z^GL#FMU+d`BdkMdo^w?@ar97#J`3e^Gz{5{H|@BjG8su##3F;V6s8+ z2FI`vtYF;WUv_P~Uz5|u9|SH)H9o$+8VccqeSLi}>$>b@EeXGYmS(#|a*LTkR7eU1 zy&crU&&BeAo9%xHHl2P=9UaOSr~khq*X{!cd*`OFDkGVzu2rTEdixG!WwKY=Uu+<( zSo%+6D$?)X4ed-^(1{IoqyG@NzxxyNF8&fZ4O?HO?q!w~Tle}EG6vuDWk~t*^70km zAlT$Ujq?$Q@q;QiGkDuY^hpsRaAT4y3?z5y0!JhU$9cheyzDa}$ih#N6Tk%BY2rM5R1Yx5RAwLP$6~=JlFU zR<;h__vgZ(u1+=%S6~!kVv-uwRqZ|zxAuzUFlZJrZ0CdfAohL6(Fa+V^U<>-U!H4y z0={GeFl_W^y{2IxDJpckU(mx|1P3;LF}wy&y7-@e6^Mg3vXrdVwkNP$HQW@34z4tXDkb8vT$FQvTr zW&VAkE-octSBuChXakwLsbPu1y(qkEnQSeYJD# z^M{LdKGhzRbTj=+qw(Fey0JpqdG4X)+j>;xwn!5tp%+0&beQg^AN2G(5GN`Y=H>(` zmCl;7OMB(-Y%;!6#UM6aiM*N#g3_=lgRcQs#Ysokm#ut&oSd95)btu4Qy|d%Ta=lX zSxIC%l#R|+`L8+F+k+v+h?RdVEQv|gB>WNcN$Z=F)}%Sus$TD2`3oRp3=H5ZsHBUN z1@Ocy#|<#4iBAFp17j{l@%xuUr#zA9=R}>3QwY z`}G)`Yt2#g2#R9?jRtfAkk0pk6kiBA;I}%}0m2M8F_bN0nwk{7dI67~OP8Y0iFLt| za$PUwAqoHRmrJ7-A#iD>|Cvi`Z@R||KEr$rG-%W1j6{oHiQHOasSuWSm23&DsJ zhcfa%(-h)u-(6JxCdd^*qf`zx-W%)xIL>-sog0ns8vrHxybWvsJyqk+n$O4Gwf5SVeJdEQISM|Ei37I zD!xBH`EM>>^rb{QRc@hNBEC2Yz^uOHXr~uolXKDWk-U<LzKD}78c;ukR<^?G&MU5FfrroV=R}8iwnR32c~Sa zw8I2IgyV73oMF)K{rsd}Uk=_Y#tL^G1hf?Gb2I2U&l9(i%I0RZ;KgFB>om|bbS5A; z0UQD?tTcBB1kl>sSvMI$DKf|5e13eYg{k_y`}XbI>B0=OGJpUbZEXo`gyYIVQc`Id z8BjOdECg<48Gfwwi{9+3Swxwzj$#FJ`8vSAqbsZ|xstVxna|?F?%^)mx|iUXI?Q3s6q%XBp;c zK!tE|lVC0{K*7W)ncp>H!!!fhYwecm+v28U`TlJ&IH742iQ`(J3FO82Gd{+k&~B~PJr68 zS{wQR6;cAx=*wqe;o<&)f!x1pTy{E(!-y-ctgLWw zaHKo+&|SQEv8Sg8tPXCFy6#i2E2&Y#oVihA-o1MV`pz$2Ww=aElk-jwu$Eg())>@5 z1HkulFX?J&SsObayb{j%wf8DiT{1AkFk5!uV?0`*va=ZRa|5nY!YIOQx&;&724JsM zQ*af8k52ZGkbO!CDnz=%z9UIFl+4qO9InDf8On|);4Uo(1Qu*ZCJr4&_ei9z7K5B)vWr9d-5J(ClK|jq}0^bi$&cn zkd3ygpietYcnOB5CLD5p3E%BRkDt5$@=x1MFT#3KcXn<7{iV>}Td!oaGXk%*N1M+~ z5SAwxg+xRaLEy8u=R4N680F**96^N;$Z|auc5T@x_(ybSCPTbpNa)KVBgsR2RT0F8 zPF@)SY|OwC3HcP)@|%}Y9b9ftpPq6Ge}rOBFE#!y3k?m8kEczAKhh%)9g;FLe@sm! zLx&$_qgX-UJ(O@8>+khrxN+$!L$=y_dNb>?IHcvRh*`DUh($D18*w{TFWoMAs)_Eu zGC(BtfMtwBfz|u+{rg*+EJj^Z)BA;mg>et1PSGL`U+-|pnaya-EZnVu6wetw5*N=Bh%)JYLd zUxKDy3ZtElFfRG|rQ6?+owQc+^LuW9F;c0H9>HDvZyhfgnpqGPAb7U6wyGdNF?18Z z%|;=#-_ZCvISHwlO1otSvJzBJ+}zxdT4qlMus5Fy%SJ&ve|)JfmD*MY83RBBj16p% z#auP=or%|fNOw&2KIoK^Cr=u9vZcdox|TO+wA|N1+JAxNNz|&b`>OWgTQ>x-Sd=I7 z0)$MMoXcQ~1avZ8?d9XsENb@X6B7QvIcG5SVi*jjrMcM$s9&hYRS<$fm!t0}s`4*8 z&5f6ZT+-6Du#e!4o?yCL$b_trFLrIejq`CLmi70WjFKUb_FA!gFgP>;+EH`ss zP5+MsK=1s&2!P#sqXHcT1qE=B&XGmnQ&OSu8$yw96_Eo)hIZiin9ex>Pn1Icx@?lS zbeB0W4}KGev+X+F&09+*IAnunx>-~?+cM*n-E{!+XGLh5(ah_}%nre&HtgwL*vui!p0S{46 z_VvlEM6=~n?9Tl$)9hWhVM)9l#oF9toG-ocZAkMVpO_e_1wN`Fg6(wBwhu1&`{)@A zp!RZFg}x$ha2RgaMI2^%S78U{ylXT$1@}7*tFrwJw~uY~K5qUVQE%6*yIy;JDf1#n z;>-S!4>7I%{;frqfvB7ddnkG3^(v^o4C?&J|0F-#YL9(AJU zc5O5j+g~C%$qpX80X1sT+Sv7p!3!t7%m*Uw9p+SEGY95w`{$Asm>|HcdoE3)#h?N< zPM)OwtG?5@0?o$5&U1pVeR6Aw?!*W1<$cM8H4apJs3}JIB5cH*SUfp05)2Ob(}A#K zmxI5U5zICIZBh4_Njgt9pinSn7Hjh61gJsTg*n>|E})Ojf@$;pOkdlGIkKC*fOgoOHCwVp-mSYGh&^G zSx+AlviI|=QGq!0UB#T^?mb{w~6B<@Kg>9t;1jro{W_6gt2hWcC z@hT{A1aah~<-|tNTy~bl-f|reT#~QKhtP~oCd@qUWCdXW91S=#obI46PxZ>JL7cTC zHl(e(-Fv$7z|Pvz$C~m(r~3}9#B4-^>9n*htJo-W7Hi&Y3Bp6gjBawKGW zwU95oF;Qe?^z-9D|2-k4kM)@E;|<5^ntP3jO9x(DbKS^eHcBbdxq5jc zcANZ66?-CAb#IoU)jmfp^pB0uzwb8AKAB)uK7PBmqP*N4Xo9|DHo~Rg!#f(%av^f^ zQDytrg8=!UU@|gHX=sqI#M*}!>Su`>0!_zDd}CwqLSh~Fmde1*BrO|*Ex`KMM5mgg zj~rrbjy=gv&K;+8gE)|9Qc2HV3z_U;&y8OWQSN*kd{h->K)BEKHJvJYZn&QvQ7ypF zZzG%J8nnc`!q8=i(cDjC;w>AP>-X0{b@(hMHSR>QgUv1;(bUv5Bzo~`p*@jzx~3+9 z11+L7UoIOcxzGzcccY|4ry=;XCTt-%Taa+rb`Tch;2>r?WP4s<*Y(yAU7t2hl5H-4 zgs=R+rfjx=ZZ=K5I@unRp%}vP$4s-qc$bruqBxlk-_NXqNKv8;^Jv(lhANDnyrLp! zS@MtREO6Jn2#%{Eyg^^e+QA{;m#0a)L1y^sZ{NenoL~KmFA%H$ z-~2Y5%CyRB6!I)az@orj!ni+qo+bakvITVg$9;j;)m?lb?S0$?V1A!r)9)X$B28;O zvPI`D*k7OCwKZiIw^f6f4!qHIU}FK}#AvBa3-0ahX;GYY8@_2uv6Q}3Eu-+mv}O7R zoDzUM8WR(9s>*OB8~ZVEb1`~G#Uk2`{Y&A_pT%*IusCE0gsW%v8HoYj=h}CV2W`(6 zf~m0uVs8oMm$K)I{3eCx(rzPLZ|(9W;ekxAnD$y=Z1qP6VTAy%-{E@KUD>_4r1`|! zzsLl|y%X8$|8nJ!ofk3br+lXwe|XiA~HYE z&PLh}8$C8VL;tCmjr8i3p)&UB8n+Rn{C6H{P@Sw!gT_m}Kk}r4a5UoC-)I^~8vt;j zq5}E>xTGz_W&Q|Z%9LTs3XPlS^Y+6~I^QCT=bdTatX#HYdrny*tdKt_Wqh|BMq3Bt zBf#LR_Ni>RRLeI%p52y*UM7c+2GW?A$uEy}V+{?my~~RDPIk)O7^6=`AqFYm{Z25< z!tQSgWnE#=HGw7U!m?YKi*31F;Mjiw@sIBjz7Az$L=Kg2;|yBGX+C4;pxTVC){ z9vXE8MHBM&t77-w6e}E8oodGpV$sfnd23JrJVhtx6i?*sdprk$?fVYX9MWK>j-*-p za6qMFLU7AkjMPfSdB5&E`}w@;*ptQ=EP9qcUEYqS{8TLW8=c^rD$C{c62INRtk zcc(ky(aQe-w44SVw`ynnmxH^T%W>RhE@0dnl`tQ3(kN6jwaC zuz)@t>nnOvsiLrw{nmhkn*ZdaXa0vG=yZue-WnVn=(sBoBkls8QK1XS{5fmEEWalz z#X4Wr;xNX@9k3}0g&YFj;621R@!NnQ`=pTKb6CNNtIiLoRHx7U7?g;38^zT56HOi% zDR3ZDk3Ay2FrbelQa;9t=d}Ro%PQ$M3<+$j&-?@`ak@BrFE4aM;AFDCoatm$tf!}^ zZBIz8K0R50zyGVJ!=lG`24D5r#SX>;l8U|S)|X8@*$AFyI9~BhY7I0F@YpupV^X~m z7Z)cbLkWD9dQ;9(CxZcn#60>VCybD0M`Rk-aAqitQ1Au*r5ZDwgR_zHt|s|6ed{6z zGj+pKi0E^z;3H|g|LrT7Wo?-(<7k?IvK)v$FrSr~)5coDM)jDx>i26ZvCq!{(IyPQ z5g(wRg|qsf!J_w=02@o-#zMGl(j(n;Wy9ZUajoa8v=I7ZxIw^i*2F$$B6)Rq%PXH6 zuv_`J@$obt^lwz1D-p!M5&`{Oc<1^I9=2%IB?vR2nuq1jXsw*qkz-6we((T4?cTUW z*2XI5_vBZ~kF9HSft09>>oTKxA zL}Fh5krSgRdN_9P6tROWk-==vTQeFny+S`P>2(o?r`T3s;HEn%oNc@A2b8KA?K3s_f0RYou~qScN9(e~C=5d;tVJ5BN*N;6R; zMYCWbxOertYnrklY(ZFA&UaBGh;GK`cCp>?rrB$KGYtBrPd;&6cRT+R!dA+^p2Wuc z2vI{;H5IYBeys5l>iO&auX0+(d@zUK_7yqsX!61fXd}gU_KaD41={Kl5b>{bj=HtB zv0sov5T(wg$Y7K8u~;FbKw-PL?}@DR3_Fov&VHZcn!=C!z{oEP8EUH*p_K>{owt9O zy3V*+3UT4wE&N4Vk$g7IlPDXHcK?2EM~GY+JdN)X z^Mcm1xRp&`6DdJd-SupHGzIDjgwR;tX(`fd>Jk(OZBx8fAtPES>0FGLmE;Gz+0*lg z3q&W=J!2vUV-zbRW47(-U?Zft&%zMqPfmmu{r#q&Ij}iy1_-NWC$QO%P*6b$a+A!} zVvYTd$gz3rR^pp7b8=N9^@MhLKKt<@E@=<+x^&mirb-!{cgUO6l14jc5K=fIYr=8Q zdYt%XYsP(UksC)d6`-=bjjm6)K1PkwAJ9a=$*Go!P{z+EOwyt4HiGC<&Z;QM=sgSVIjK>H{YP8~e?W z0;w|@+(LqJ_l!r1q)O5!=`m*)4l4>hMnSI^J>RnhCM%w6wA9qok*#L9Yd?Qn3R~E2 zQWP-Jgrgl0rVls#`!*DEQ6(sR6H|HCte$(=fULu95PY>%9hv;&`0GTYV`F3S_!lAj z!SgtRMS1tddf}r}e55TaUf5$8rxfASCs{K0S~V|V-q{`1)zfQr1*N;YH@5#%-F4d9 z9mV716&MAxdUuCddq>B~234=TJx1XY%Ep9v)5N~C2R%?UbPhE~5pxq-#2-T(Q;g+u zCm{r}$OvN)?xhmTh8Mp+_=DGCXPx;imBW%!-I~@K zps3%zfQ380jcXoXryn_sK3k>MOda73gxq{iGJU&Au>cW88myP$M9N~OsONJ1uZGff9(VZ`1wGq&H0G6AZIMY_n zK@W+p;RfyyK=fjo+N{Dhlhut^_3kh{Pyzv`dla9BkitVJ z<2Bt0M5ZstO|q5cWrOspu=~KJ+k3dhO`p2{!$MzQf4ndcbuQU|HVCJ>MvgKb;<56> zuN(!olgSTXUD@r20*wXgDuH|MGhyrm=5$_&@V}`)gaE7tfN{GjWtxGNb>aK>E1MSy zEOS5D6Km`0-kUhCCj+6`R%e9d+gDarItxEN38tBlkdVK3sRwp~6KUw zW@aq_lwsAC1_?h%4<5Y3E}NPOlGPQ593_3NZD#l^-_31$VL`_{MN@7X1muBu_*`q` z z5QG7Te_Dhz{}JjEATK3J5p`!_s%bDvIj4j6@%P^WkPgDCB`C%7c~6{(1*)6A3#>k& z`ihwR{CuTPzX{agdHIa2gP;{?ZW|NV zxq5j?OcsL?E|?64oSkde)5oXOpaS!5oI^ffYG!72tPqV#X)yui05kQ$>&6Ya)yDdI zxyB`sO%z}<=$walsAs~$4z?P@6;kt&<)9Lz#+qwi4_$5 zT&E2vvG-s`M#ke6*;in+BPS!k-hOy&u~J3KV5~!AVSPgbVNv0@{wf-9=FJh~WNu-x zxU>X50TA(;f5xFa7}4rVeBI*mQ9e>+CFeWb|GXlluC9*%Bl_e?9~@vtZ$J4J42wji z&4lrHsiWWJmp6&wv>TyQ?fd67om{+6YGvAvYa;ZkA&{I7S68N=OCYQdahR>GX=-^8 zVU4`p9L9f}z-hI@Lc$qiBEyuYPoLJdnghV*tu%jPP5kFuB)0V_IacFx+t!{_*YO{{ zy}ZH-e=))YJ3qrnD7VZ~0HLj5I}Jyuw~qll^k|NXIwT3gk=8tXI@~!l^poCo^XJc@ z-rt|b#t3@MB5Z=zW|Qr;%Cqj{JBD5_!C^d*eHR8Fv|N8aSX?EinlNf>f?s;Up*e+i zc6P!Uo}l&oB#=3B^VYhLMLB{&L1T!Kj*hFC^4dFNXwZQy7b_qj;CYWdx1y%ztSGq% z77O74xaxW`eMrld{_uMhop<2Kwz%D4H7I8v78MD8t!x7QB!^5qfU3mJR0R|n5<`7! zYisWVu=m@dWI!sSOB8}r&wVplB=}3kXy=YPJGr7o+B7R+y6Ta7;|fBReZ=0uJHR z-^-E1k92>{fKe$u;CC5N7*NMRn~VWbeHarH9j%-O{+sEksVDuQ*q?g~t{))eh4ohs z-zuV98342Murmz%aZ-OyTYh$Z?rI%v?Tf&rf}6Vuj>>}!W-XlDW+llETH!nhtD55i z6G;gP%iyAZmO^|MPHt*vQfzN+Rm>Bb2SrWUEtIvjK&eiAIGL7~mJ90Z6u5!@{_M!hwzgR?NGF)U4-^cWq`+XSC97$jfs=-k zgkK*K0(WQLPO3k=+W;~>Gjm*L5?Q^K2fqQ5&vq-w$3(Ze&lNnGhw{K=kVtuWiQq3V zCEg)R0Bw=-^K-&aihR47H%J&LBGo2hIgigoUaD$FAUSOB{+uYdv_nmJiof7sH>F@l z8U|jc+pz=|?&Y*{JYhwPdmk@yU^MvigDhM95;Zd5cKy%4@1hcQW@Vp<-E92oyU~nd R_>mtXZFN1hG8OCF{{uRn0J8u9 literal 16265 zcmd73cTkgExG$PSqZoP#pwdyg0s;z1@1RHvRXS1>klq9&(yNF9q6ku@3kWEp6r~rX zccg<-l-{JBC*QaCKKtJL$DKKIX3lZOQ9|uL3lU6UoZ~m9gemP`$asLCC!> z`b3J;yJ?|)ZMjA$R)>WY@2R5vh^7J+;4V&sWT$WA3h&&hdhc8Gonp;JNR_pl)3Fx!$^fJ8?&@l&nIk*vs#@9V$zaiylw!KXw0O^n&@++rSHtOrLuCE_l#| zW{REsq;8~9#%=JMWJZ%HO3CjdMGptRyL>%Xt4|_Cpkm(e{Q8h195=S!*xgk*`Tf-KH^D6j*s1{|Q-L6brhF#L_-wY%L6PyEgAG&R) zN;*7gA1{+$VU{SfEjAP?v#D7r6!9(9tCl8W)w?N4i)6(iSc5Sv!J5cm3LZR#9GaqF z_F!)#`I^)A`ubvTjz%n#tO9UiIDDVK=~_e>JLs~y{h-}G7;xBjSw zR3L6>Y~0F~=%Tp0 z`#RugFW_i{IpGeidiM9FzgtVg!^1d4o)vnm+V>g@%Va|YwdJ?|0{z3oLyc4kZmGoh z_(}HrdelPhuC5P$R}7L2ntgQ5wjX?(a^0(m$pUHB$`@B*Kghwsk^j1e?e=?zJpJPF zmD=N5=qKWD(W2twN57o{sO`6ZkB05LZZ8i{&!kHFj(l*6J-sW*!@|Rp@3S$lEoG!# zS7K3*I&lJaFYNqvxixAzUX2_i9M%ww{cbc)0HFZr&g!xkB@e06-ii_6~0qzNVxp`&X~o`$G5q3>7AxCRe5Y6SNfk?28G{o<9$ZCF;{R^Y33B=4Gt^J^G>4KG;x6>C`>p>67hk zZE5e{JC0`jcnDhn0L8~SPQD^h6G}&nt8li4t-@9k`=$TGsF#0OI zoSmJWX^y>q{kjgafsY2k1$At|NGlI^#bAtIqz8!NHqrlt&zz}<SnkY?_jo$T4nz$Hpcy;ppgSy{J!Pv9y_Ke{0aJ zG+M6iR`#o{D}rwjjrDl4k`l+!t5;8k4|5~DJomSM(_JVe}8b6jiSBOwU{dA_U7*m(Y4(lRUQQr$SVzPicz$!`Y*k; zpQ31lw2INc7?t$&X!SB&SfxV4!eFP2X`Gh^E}i_miK9`n8-923T-|Q?8zht6pFe-Z z#KdyD$j)9lN$R_Hr9%_B>kVPm*FjmHVOo@dNO(0^Xvof#Wl-b!+bHN1=_;kVod-Ic z_ES($eRrmE9QpdeR2Xv!m2Yz^+hnovN7vQKFJ=$-8XKNHrAbSQ5QHo^H846dVi{}4 znD(f*&`^j6pA?0{Cnw*AFlpE?Ie+aw%dA4C3_h8@!f?&zZNTx7_m!xv!@pa;hg*Z$ zcY2hAPygBS{5_H)9l*q_B~qss+KO|Pa+@a^*xlQUh<*F^E#q}hVFxrllf10#7s!!P zNBHn)`Q43fW#&VzaVK)I@M`xzV`l{&rd5l~YL6bBW};}1e9=PP{v1ore)TG?+!wX$ z*FWL$2)MO4)Xk~I{sQU4O&xUx)ry~my$dA!iy2q7w6$GhKk0QeY}U(QRsU|4dZwXN zD!B0mY=pJ|Z6X&w4V?nID<%y^^j|MQv-)4_90p95x0tLjUHgg9hV7wPT;Sy7C%-ePZgonUDCnff=iy;EXUJOV6h={U%$%HQetIkdYD_*S-Ekw=_>&HmQncl6=_+P0t<* zWV&r<>eYa$$w>-k=1xm^cZH^&-u(15+dX8OS?xp1zncHNk&R96GhM?(Nr?b0w9G={ zlv}p8wx`h&n_cn-Hjgxs4T}cv?wp~apr>!iNCTLlZD2rzp8ff=j4KymnVprDRYH-A z;LT1;W8vViu@+mQSUNH2Km5)*`KqHh>@(KfgvuUW>cW0pJ zwul~M8tQ2OK>5I|#_xrfnAT6o=<`<{(op567QOwBsHw$VXI13nP}sD*odAd9XKJPQ ze~5o0Ns~h#lzOjOj`w1(*bnfP2OE+F0#>rOp%<~&mx17ewH9`>hg`~%6;n^X`g^$S zCv<2N6R~S|duC&je~YiHtAmDW_z~qis-Uc_gQV?jOW>_=H)=F=w(ShEv+<8^IRyp0HBV{qMJ1USH}9jppRgt2 zCMzQq?LU5)qy}A0gEVu)$>iKh8Go>JNm*Grcx-oVrgZ(&N{#$b)FR^7=m&n3HfB;O z<{YD_Gotx;cb<~t zmw6m5L<>GH4+wU}HbMx~V z@6EW;<{$m{x0#rjrVDa&xoU{vmhn|uh@`8IM)jej6Mh@t0AD~Uy!qhsd1CX`A*zxnxf0RU@`1iq2ot5w|DJv_RdLYqX zU0uC)L78FPYr;QVwe(>_IOUK6Q=P1=?DPy2e29f`kxNG9LzOO8(4Q}&TRv#FPOZqczLM~H z_wevMz*o*Efq0nQ70v zj1JF}syW=gy>-VU(65H2PxIj5fTp&}Yx7F+v%>t(tk^VJbjNFHW{vs4K-3R2t4Wlg z?^vhw@mRhtud#>EgRMQch7V1azO7F$wj4lF;KAgk?D_QBo_#4`A@^F!|7L3z=F`>0 zkK7*@vb8$e;?Kn2l#u1AmOSzKWs?!WpRi-$TEf!%%R+u0P&ZBXR<8r@ z{@s&(1JY%GXNCXu2aS%mLCGg3BGcDrOBrcsf}7B(2hhUrK^eH>?(8gss=m{-u1sot z?u8Rzt=tm7&2HuC8Nf0-hGjObosIDgl$4a2%CYyMvO}iO@)@Lg(SD7vW0(>s*>mfZ z%qbYi3nmfL=QgOm+6M)h*?SVXOv9nup?27pMSQ1{6>Hv)sr1`(%~XnMC_Hb~{5VC* zzbZ3xrtZ;+H9=9)p{6FKs+cDZBjtAiQ#5>wiIi7Ui`*2b1su=pw<lx32m;HyMqWYc;x2e4e4yQx0&XC0($zccV1Z}w_b4$iTACMOLw zH~)l9`%&%tAuQ}P6@cJhx~fl~JxfVRx$dz@sHA`$m-0P$SnE4;`o+#8Z*U?^K_0c#Fp{h-ph)Twh$fmzP!;XJ?Klf+RFKQQj0xMbDVLorP9~0-|l0VI#`uwAg86cu(!7-WYZzw7i+F&xh#3*O3&cnPF_yx)6N5n zD%2kr9#)_EMKrJN+H|8zgtq_A53f?Q#u?$cKFBZ`3mXd@pG&j1k57f;=yp@g`4WUw zq|u+@GTTNSTz)69v9Ym(qoXbj-i9;(rMG7apkFz3?$6k&!`aqnT8;jH7jo7v(ABTA zZD4uz=*Ke`BW|dvX)1;#cdecwB`s1yCrKqdk#O6h9%L66KX(Z&c;l%B;PjDdF1!%2 zy&f{-C?ffk1-t3F%%Z4RPft%aQHI*M}HMW?UAQ$&K zUy2lXEDe}^@>G;$W@cn$?2cFb>r}JXw0c!nBl^8?;x(snKHZr)A+mNX%QxBFfSF#R8<>c5x5m1b4E4#ARN2AfDaQ{b@BH9i)mNGqjImcAL5xMm8iH-T3s_2$K6Uj4Ue+LGpV zh&7jq8h;gPdiq@NWWA)_k3UoGab&lA#wuNQA-XCaoQ8zDv`9AH+`gEiro!)*K>eII z4O(#v#a))Ap&^6D!kc0ujil-do3!bua+w6%k5%pqRDYNjJa82{y?ey#8yk(iIW_Li^NMgKom&hGiv3C#d8svuFr40pe?7Ql&hJQLSC!@DLE=as07 zC2Xfr)m|R{IDnssxa6`74q)cBd)KV3t#NppC6pcIYBfUe0xvJ|PSZkfPKDQ~4ie*Z z`e*0sTKmDv^+%|NkxIZ4mX^~QicxHAY-%@di1^g&AeBbZ&@e1}lDxvV9xc^aHe!VL zg@l9xjt{*^Cl+xC!N`~xQexQ+0>k+4jN|`BLjI4+SKTxJrA0}i3k}Pf%$nlZl}lV< zuL0S5_y*5AlT>sjUb+pGYX^F&4l*l|#@yC+r%RsP=df|S+PBnosKnyYqepLkCYpYi zWO{1Kksx_$BkSEod61$s>u&H@yT^WnKh?X_QS(nCB5*gNUp$SNR?Tw&t*}h9F8B=f zIH)mqeY!Igb6Frz`J|*oZ)S zB4|-Pg+Q)$SEnou1*vt^-h8drND)uKuv8S`0sUmfKLY_oPh3o_dV9pa#}%-NnfDZW zY#Q*K+}QV5Vr6M*h&9idCejNs#vq(JL2?ml8FY@(Zdm4-nHe`%SBhH%{pIDmckhrV zPL(^3(iR;8fO`ln8{}G%4h8X3eoNESfvYMvZz>F#TY!{Wd$^sf(wnQr&T#zb#A(0_ z!tTE`>xB}Sk3m(kz%<-eQfgW%uDy&wu7w`N@rjA;0XWx8pb1V9_aNvdEri++dPP0T z(=q7`*}Jb#wZ=59S3sq_h%R>+ey1;H_mG#z9EyiF>3hH)pnvry6%^M{Em0oqRt6kDthOR1B^7yt;Erj*DEI8} zYsyo0?}7{kZ*07Illw*6pA-zsd@e$Zt}Z6(%~l|qIP3h6(a}+7eh_wEY=#q`IYWs7 zLF$5DhX4Isgbzp0LycFWk2VDb1uYLBGhSb%e1QYvBKFy1NzW&+u#m4%K}XJygq-|2 zFAEo!N+HBe)o0qY=Mmf&b%v`!XI9LZ{;8m5-u3jnA!dooo9NbUI*fj;{V@)?s2|;&f}oH;5KZ zH$uCqu9nmT#eQ1kWNN>1ER!Nx%kMDehjkBb-@ZK?`*-hq%5};*Z3-!X2=ZZMitj#* zKOp7}`R{A}0tMAz2L#~rmnSM+W|O@Nzd%@HKGI9JHR76mRDL2Bw7kZPw- zp43EU>0vd|t2cWW+(SLgB~HLC*(cRhR?=UtL**mZN!RN5xVZ%dsqv?_fiU1M<;2!= z@$ft?28{H0w@9m47j*ia0@%-M5 zG2ur8k^ik4QusyCeRDDp)lO%yGUZAj9l5%`%Ru$?^Z++8Q~>MailU+-k?&n5VpXJz zi;KZ!KE6AY;Lyx$F7%%6PK}Shz|A5ef+YrgV@)uD_Oo(wM&nNqPx4RKLS`B~;h^Dw ztB7hy?cD)3UgfbA+Cmqg98Le_AC2hAlP6u7%H+wlz$mH9rm4W3LHER`2G;oRN9NTK z6rvDFFVHv|q>n;FLP~*-S(^NEuYzEr$NU0@J;jg&qS=Tolx$6WZLM@vAckc?#lnJ> z=E>{VodgLf0a)Z$Irxv^1^}12*;yzXRzP(@d|pYC1h`=g+!xx)W#h`ac4hRFe7NnD zqT3%b8kx?Y2P?}kMRDzi*95d7d(e`=$ePX0LaAhn`TyMnKdg3d{(MTvv-o(>sG6A> za}nBSGBPq&X9w?wK$K>R*9FLfN2MAZ5<;NcIhWMb)Xb3sXm}rTZEY=#oO$3bWgmLz z^dk(*J>x2OZed|ujtSf`0W6!^@!+|r&z|+!TGUN>a|VN%4tawLM?sY$BPZ97ea$K# zD$JDwd;*x#_7SaP*ZV#E=f|h#hm?7U41{Io*Vl;Cr04o7?Ddl@A-qBT z3L^#ztt*cqA|ZKo&B;`KlMesU`y;%&69kgm76&r_ibc@k@&Qp97#L);tpe88-Ob0e z;_6JzdM2UsGlhX>s za4kx1$)GLC1(B~~bdAKv7ynM;@bLD;Plc!Gu=b+u{fWVNai-x9a^{Xo#c+Nxw zHD`nVty{P3BBQKN2K?qiv_r$(SzV;jh}o;kowbAUXbCToiX4DTgf1k_|ewJ`dFb{bUg* z6EZtLKVWfu3~Ta*420n<(8jc{VRHS!$%Ax1M@97?GR^SF$PrNFf0P(SBxsFWtEhGBPre&oHC(IqL4+yQ3eRel~*dmq%+*Y8hf< zbP{?eDKW7;I!r$i!;)Y}35vy;1lt?2$E4@4g3!GhsEy3xfjy8r34#f@GXU(gBg#R- z9K?y(^=iKJb@9_vr=#oVD>6|kj>2a_CxtqyoqWCVPl=AiZ^I*kV+{He&`_Z~I^g5pl=X_ETkL-_-uq@@?S2&p>*=1h0># zG5(pKPki;t_(ND+V&W-c8ald-NJ3ME;&|rF8No32GsMI*1z_6u@z_1`_3?SpSs)6w zDpZEGDtO;j6%7q94-b(7vmYMfDW5+1-nnz3m<%g(kG8yu^4MmV-wW{m_wV1qxpg(XBQV$b@e%2SfDLUZSBC_4{2#_EiHKTkQs~+ z^y^fB9P+;M?XIXftb?f(4}OdnY!Zl((gyahj~!XZY%s2n5F8u~(8cgqr-L6iq?9k) zwQC>lA5)f8nPO;YY1i+;{<@CU`1?XuLS1>cNewsV=H>NM8k%qxReVCi-tI1+;X8%% zIn*>X!d`!jP4&Q;Qd3b0Pi&g~8WDX~h(lcbB_VpSL2b2whJey1-ooSrJ)0)1DozGk zrQF49*YtgJnvUdPh42uAnO_+|i|VJ24iBH)g%<@Qi;GtXHiG}-@$qp#5ui(Gzz1Lf zh)DC*-pY}`5mtOE53w$tm7N_!FZfl>X0olb)63a;lJFe|&!?v?;jun{mRJ&-1H318 z{kmHZ;TK|@olsvz4lrHxHxl{QwzlT{?tt(`&A{L=Z}q63!qP8Y`~_(oAY(Z{zt3|= z?32|&)Yrn!#1TRYfJUaIL-#R}_X-%i85F|9*hpxi;re(#-XrC`BqFP(Xu%B&tvrPG zw{L_!AS>e)1gUPxj}>4PH|&L!l$77u%d8KS+@!AX)EO;SgTr~CI#yymJ3ak%uO$}| zqhnz3YkAp0w;F$wQ%uYUHsOGijYp;o%=oeCWu{^|bMv&sCbe`_%W=@?NPByfqylanHy4>EtrbVzEiPcbG(-6WM&TUADXtnO1rpp z4|y-b(mhE@`Mtc{v9S4vvOK+T?>H|HaYM1=Rr;f|$Ztr&8jAKu7ykXR#ij7ZEWHptT{SffU?~A*kQf{t92$;4zopI{b?~oV`R;P8|Sz9;&}*PO$W0Rcc6_= zPIdttD=#nq@yTm{Z*MrBnu_Y{Zn5VTAt88Hn&Y2=jwgB2$h^1u^*;d}2g9<#4`6s{ zG@5>@oxk?ZRNpI4Ejoc^{)Q1GR&zA%{qbr&`DhE|uh@f4t{ZO%$m+_gSO?(4d}n#m z^uAdA1OkBlG65|2((U&pmyNE62e8TM+<0rXkYgx^?xTW-F{yGdt*9_5F-Hc~CUz%h zNk9r62)92?N8KyN;(hP0&$cp<<-V0U$eg>#VI374+dMZ}8HAJhtz}?fHz$O8dBTo3 zJiVD<#X#rT@TjZiN&BCsW;=4w{=ZDk==WyG$=25|UT`yqAitGfypbqzxY-L-`P@|p zdaIg9jEag%og~fq^T`Pb+3bm}A;ff4)YNofytvFL71boyonWr;U&9Qvw-f&vg=b2W z4SKM>93cXaFm?Wm8t){)2N1Bp?jm9RhKQYcnJVRPRPezATqUOc@I9~@K|mND9VO;A zf8lhJHwV#o0JA~-rk`6%x|u+^9tX23G$0@1@U?`GYq4o{*2A-!hzTM+Ii&Xy`w_oc zO^01K4|D}kXM)~4xkzk7C0qRUi;D+cX7kN*C^;X|huu3ElmN|LRYj%iAcl#V8Aw_H z436M1^xGWVh!5z|W4*6H9ttg3dV!pDb!l!>3hN@F(OoSVrTHEAj8kKNcJ=_wHX9G~ zT!RcPliuV$F7U`cq&^~n92+t$utfhWPPY5~>4t5USgkv;)pmEJd?8&vtmEgdD}6ebC~+{XE+bO%(KuH(6N(DG%mgpgU)y^68BK z{QC8)3iEJVJgWFGQ5{b}R~?BB4hR6QT(y&$Hr{(GM%YwHsVbO__QMO+M-QDYQ(@0G?*1 zqYFzuuY1Od7pewjx&1)l84?l)82N$em6k6<5AFogKir#p?)7JEVR12%B-{J#SB`o4 zOT75%TlYw}#+eh-@dWt`ya|O_f~>~D0iwJg0N!vJpmshSJiKPUzeTv^y}Y~5QIsv==6<*ktuZP#$dKENRg8H8-2$4#S)_<48&gR;Dy03wMw zCqhLCghCzUeAm*QmM5&FN6&$Z=4J&uYPWy)X)Ks?#(6-GN71P7bHk18;A_)DppcjfoL??+TO;&>|F? zYoLpHEDs4TxHvi-TE$8Zfcyn6NN`vf4BfO%{MNOg=EdQ~HpOL0W@O)~hdwCo{f}va zDin=xfB_!m+@(EO2h{OflAt`5=fC7(zze2@E?*`ib`IWMoNkQjl-lQ8ew+YmIgbVf!;&uiEeeqO2BhnmkVS&bcJ1}^}T!(bS6Y=FvF9Y=6gRk}kB7_2#&v>1D6 zcO=edEo-Q(J_*H_%^7aFLf&pil*R=J{Z^_(o$EYG#m4pI|_bLs-Hni`?r8XTO^1(&Wl zj$~yZZY-Z641P}5VoEA1BzNh2e0;PUnZ#GOw)_|Ka?>4zxsV|Rt{6jD$4XOW(i$?R9L)Rku0fD{L<{M{jcyAQ?<$r!G9}h#DuZi}bn?AS; zCJy+TU)Mok%s&$Entl+Nl#d6yp!kAbXl!ik7MD4&mB?BmiJdXGm^Oe_jMwhJ>Kp!t z*(qej@!vMqpcp_A*(0%_k;K*wB}|C|r|OT=DCFed<09rRvVmh-XB)?V;;Vy4s8W)@_qQk#m~z zR((MIWYBaA;J^?aLC_P-R+w~H*Ydilg)dz*AhF$*HOU0>05Jy}yhE^(_a_8zKVd>G z_GA-NZLUm63kqtrsa=SQiTOQP%uJ^JhZ}dBXxYyCv z&N2Zq$1t__A0sitOwtdH{X)J`sz9w%OdX1wnmU{8uVQV+MVem5nd*%ig}%+Y3w2^V zLy>roMF&z+`qlctZ6|^?tHQBR{Pl}1Nqf3db1s2-S46qbwP6+L0+H=RCnzuJ`qHs5 z$11vnGS_2D;%x~nVaDh30GJayL^m!ZK4rBeXAn)>UwSPGV*xxh4`9xg@^#axNIi*Y z%ua}=w9E9#MCrX*>ScSFkjRi1?QvfkP*=0DvSKGqipoDJhg->tCY8$6ClitUCxmiM zvH;~pQqo*$`{EdLuSl5n6*oqc;h#IeqQjern1XurRX9T4`VHFUu*`i8A#QAj8S@$T z)F!AaZ>OBQBf~8I)^xSWwEg;Jx7OX>e&gkWOtG+LjI#U;4FC4M5ZH}273S2HsZg58yNx&L7zLoIykC*X>kZxrBv@JUC4q<6=<3Z<#9 z+y>)wDQAK@eV%bn_(OCO0;;mI1X;mXfIxw^S&p;JIy`)=k3^GV(> zRgW1>(oxYd%^9Ngv6WldoaqvsFtcMm3YintC+=%Q{`thsB4A-{R!u<>lfc`m(UH;*=Cr`o(s$jSxL~ zZfN=?4{ho`yJl*^4~+M(!i35dc`Yn<^4BjStA)7`HKOl2H;FAAZeqonnyj>5O>tho zeqQ55A*E1w+eXJf&af~a@BBHlihBv=#6DNj5eg}qGJqr-E_X%JP8EesTO?KTrz7qT zxShNQ;g)qHDoKGDb>>MYcxhSsPv4OI6G-Ap>OXj7vIiqfuWXg&jy~o@jY)_SS9^8Z zz}+w&CqHTP=8F_MNV{df%y_-1^m!N7}LaWQKq!a^QF?v`+( zx4)Z25ooqDI%#}KVJ(a)@5nfeQ3FR~4qpR2{}0w&w5(*I`c^bLRm@#N3x_xSrkOSO zB1C;T;!CH+1zMbaCMD+zN_{s>j0V1$T<6 ztsd!r_|J={4`pU%isT}weJwSSHW_%(Zb3Cw-7mrwQ~P+4OD>koL@*J|`C4Lu*JmRe zPOPWWn72JS|Hi?x^4-_R6g<4xe?D1&DDhsKekmFPPp@5ZLP)7%=cw}Gp*W4X_obV< z_~|F-?KD+msB^1?(UoI78dCxOou$Wv$ORZ`l3?plJSV zbJLwCZ63Idike!6r~Na)v|ki-^$FJkJxj4sDPKxegdt^(Qj|ph+!9fGrE+~jsH7n7 zG3YlN^WWsp(d*w69~mAlrPb^OX*4bJdrKiBP|iSSozE0+cw}5!NUgrS8Df^V&YreA z8|jNP&f5`ku*N#WOw(O~*I>FL)e9#l&0*B#Qro(m zq*0RREzBI2W@>987LV0m3RBinfvnt#aiWuW!p_Eal*`_AGetV!2$&wtinum6I3Z7x zk}BbBgmD*;Eex@lAriF3brpfDvj8wOV0Gb01QZVU^)8U5gep|KJG3}!0Jhx^C{=d7kaW+?9RoHMq+upmODE-4nQ1N z=kv5iA8e00m4q7;7B=h68^DcL)}Uy@2xGS{y|5nyt+fOOVL|&=dYDOn-j)5@wZWn4Y}jv?Y)fULM#+~~ z*2Dg4aZ}OKhH|umaZ#v=%oWTAkz@CA^EaqmN%8Rw_~KyHpRtcY^Dhz_@VRCxl7dA= z3*n(pL!Df3LO;gFXs$O=ilFfzxbu>gWQ`*xGT+!E6VA(c(C^U*W27nFSq-cB$LIOk(Ks&m7y07{Q1R8u`>CmjI=Ac-Q3&^X++eGjLt=h z3kf|_ltq`^yXPVE4$O&MbP0@YyPb4Ig8W8$Ol0-}gfo?^moGntcO~(Ii~@4v!HfKn zM+VK&XX{}c2LCZ=vN@&7bxw^|aVs(Z`Sa(mUo-zuLmMC(8yne`qI2s5z7O6ZCdQZ_ zz-Y6)g++QkiNPBa?>YpMYDR+zAeLJFT}Aosg1yN6h9YMhDZJW{mXpjA7`S%X-S*a zKyvSwJ-99^YVhl}`(u<6`pG?(D_5u-??&xkGC8Pf9i|3>n3OO>^sb2xr29r>8Lqh!LZXGI!q>oiqv{>v~KJ?ZRikYaSfL?J{)<-6A>)JZB^g=cyi$gZP!9Xqv8-`Lr zKY1w^&2O2?{u`{8>Ag;FD!Bd(tw6mBwFyE@iwgXPpy+6lvV5ng%Xm;nrn1vfDwkN5 z&bGC;7i_S{aj4tC_JM@~L73}1_+8e8d^;fk z!at_kYl zMO~&9Ji)&T|5!#i{uFIOGy^Ia&uq|0O&0#$yDLC+2s=a3&LSW{q2WOJmS_e%T76dP zlb^E?g63zG2{&xbvciWK_yg9u^(Y*xgoA+?v;he4!I+XNyap6aHMO>H-x8Sm!VN%T zR^b0y2}k4X7;*2hU7!y-@XlXPBndnyeLg#TXE`ng1y4zN_Uu`L$HIM@d=-z!!<6e_ zcehsI3s5GlZzD5kyI^Jq$e-LrI19waYi2e9%SS5;M+oD<&EnR1Gei#~Www94Q?s+P z;Y41^P}jhKUK^;@Oct@Qg$9V+w_zb6TRUMff-9Ntx${pA@U~PJT5r?q2L^%DsLYVXCLm#oh zGqtM+mAEHK%fA3y)#?VJ0`0Q3a$DxH=r*B4UNp(hcwSq$9RTVy3fxq{E?j OgsS2#g|~9%f&T- Date: Tue, 28 Feb 2023 09:57:53 -0600 Subject: [PATCH 11/14] Update ToC Signed-off-by: Chris Doherty --- design/20230222_tinkerbell_crd_refactor.md | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/design/20230222_tinkerbell_crd_refactor.md b/design/20230222_tinkerbell_crd_refactor.md index 9ad749e..5905096 100644 --- a/design/20230222_tinkerbell_crd_refactor.md +++ b/design/20230222_tinkerbell_crd_refactor.md @@ -13,6 +13,9 @@ - [`OSIE`](#osie) - [`Template`](#template) - [`Workflow`](#workflow) + - [Workflow `Started` condition](#workflow-started-condition) + - [Workflow `Succeeded` condition](#workflow-succeeded-condition) + - [Conditions](#conditions) - [Workflow state machine](#workflow-state-machine) - [Action state machine](#action-state-machine) - [Resource validation](#resource-validation) @@ -21,8 +24,6 @@ - [Migrating from v1alpha1 to v1alpha2](#migrating-from-v1alpha1-to-v1alpha2) - [Rationale](#rationale) - [Comparison with existing resources](#comparison-with-existing-resources) - - [Use of explicit `State`, `Reason` and `Message` fields vs conditions](#use-of-explicit-state-reason-and-message-fields-vs-conditions) - - [`ActionState` and `WorkflowState`](#actionstate-and-workflowstate) - [Implementation Plan](#implementation-plan) - [Future Work](#future-work) From 05249acc2fc3bbf1072eecea4522825223c01f49 Mon Sep 17 00:00:00 2001 From: Chris Doherty Date: Mon, 6 Mar 2023 16:16:29 -0600 Subject: [PATCH 12/14] Fix object references Signed-off-by: Chris Doherty --- design/20230222_tinkerbell_crd_refactor.md | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/design/20230222_tinkerbell_crd_refactor.md b/design/20230222_tinkerbell_crd_refactor.md index 5905096..562640c 100644 --- a/design/20230222_tinkerbell_crd_refactor.md +++ b/design/20230222_tinkerbell_crd_refactor.md @@ -109,7 +109,7 @@ type HardwareSpec struct { // BMCRef references a Rufio Machine object. It exists in the current API and will not be changed // with this proposal. // +optional. - BMCRef LocalObjectReference `json:"bmcRef,omitempty"` + BMCRef corev1.LocalObjectReference `json:"bmcRef,omitempty"` } // NetworkInterface is the desired configuration for a particular network interface. @@ -169,9 +169,9 @@ type DHCP struct { // OSIE describes an OSIE to be used with a Hardware. The environment data // is dependent on the OSIE being used and should be updated with the OSIE reference object. -type Netboot struct { +type OSIE struct { // OSIERef is a reference to an OSIE object. - OSIERef LocalObjectReference `json:"osieRef,omitempty"` + OSIERef corev1.LocalObjectReference `json:"osieRef,omitempty"` // KernelParams passed to the kernel when launching the OSIE. Parameters are joined with a // space. @@ -372,11 +372,11 @@ type TemplateList struct { type WorkflowSpec struct { // HardwareRef is a reference to a Hardware resource this workflow will execute on. // If no namespace is specified the Workflow's namespace is assumed. - HardwareRef LocalObjectReference `json:"hardwareRef,omitempty"` + HardwareRef corev1.LocalObjectReference `json:"hardwareRef,omitempty"` // TemplateRef is a reference to a Template resource used to render workflow actions. // If no namespace is specified the Workflow's namespace is assumed. - TemplateRef LocalObjectReference `json:"templateRef,omitempty"` + TemplateRef corev1.LocalObjectReference `json:"templateRef,omitempty"` // TemplateData is user defined data that is injected during template rendering. The data // structure should be marshalable. @@ -504,7 +504,7 @@ When `Workflow.WorkflowStatus.StartedAt` is a non-nil value it transitions to `T A `Succeeded` condition will be used to indicate the workflow succeeded. -When `WorkflowStatus.State` transitions to `WorkflowStateSucceeded`, it will transition to a status and severity of `ConditionStatusTrue` and `ConditionSeverityInfo`. +When `WorkflowStatus.State` transitions to `WorkflowStateSucceeded`, it will transition to a status and severity of `ConditionStatusTrue` and `ConditionSeverityInfo`. When `WorkflowStatus.State` transitions to `WorkflowStateFailed`, it will transition to a status and severity of `ConditionStatusFalse` and `ConditionSeverityError`. Its `Reason` and `Message` will be propagated from the failed `ActionStatus.FailureReason` and `ActionStatus.FailureMessage`. From 41f6aca83f67913a96f64a0ee0990e133d1d60f5 Mon Sep 17 00:00:00 2001 From: Chris Doherty Date: Mon, 6 Mar 2023 16:25:05 -0600 Subject: [PATCH 13/14] Simplify OSIE definition Signed-off-by: Chris Doherty --- design/20230222_tinkerbell_crd_refactor.md | 11 ++++++----- 1 file changed, 6 insertions(+), 5 deletions(-) diff --git a/design/20230222_tinkerbell_crd_refactor.md b/design/20230222_tinkerbell_crd_refactor.md index 562640c..b69c8fb 100644 --- a/design/20230222_tinkerbell_crd_refactor.md +++ b/design/20230222_tinkerbell_crd_refactor.md @@ -96,7 +96,12 @@ type HardwareSpec struct { IPXE IPXE `json:"ipxe,omitempty"` // OSIE describes the Operating System Installation Environment to be netbooted. - OSIE OSIE `json:"osie,omitempty"` + OSIE corev1.LocalObjectReference `json:"osie,omitempty"` + + // KernelParams passed to the kernel when launching the OSIE. Parameters are joined with a + // space. + // +optional + KernelParams []string `json:"kernelParams,omitempty"` // Instance describes instance specific data that is generally unused by Tinkerbell core. // +optional @@ -173,10 +178,6 @@ type OSIE struct { // OSIERef is a reference to an OSIE object. OSIERef corev1.LocalObjectReference `json:"osieRef,omitempty"` - // KernelParams passed to the kernel when launching the OSIE. Parameters are joined with a - // space. - // +optional - KernelParams []string `json:"kernelParams,omitempty"` } // IPXE describes overrides for IPXE scripts. At least 1 option must be specified. From cb9942766a5e236999ec1a09b1e025cfb246c856 Mon Sep 17 00:00:00 2001 From: Chris Doherty Date: Tue, 7 Mar 2023 09:14:40 -0600 Subject: [PATCH 14/14] Update time fields and validation Signed-off-by: Chris Doherty --- design/20230222_tinkerbell_crd_refactor.md | 60 ++++++---------------- 1 file changed, 16 insertions(+), 44 deletions(-) diff --git a/design/20230222_tinkerbell_crd_refactor.md b/design/20230222_tinkerbell_crd_refactor.md index b69c8fb..08ac9f5 100644 --- a/design/20230222_tinkerbell_crd_refactor.md +++ b/design/20230222_tinkerbell_crd_refactor.md @@ -165,19 +165,12 @@ type DHCP struct { // +optional Timeservers []Timeserver `json:"timeservers,omitempty"` - // LeaseTime to serve. - // 24h default. + // LeaseTimeSeconds to serve. 24h default. Maximum equates to max uint32 as defined by RFC 2132 + // ยง 9.2 (https://www.rfc-editor.org/rfc/rfc2132.html#section-9.2). // +kubebuilder:default=86400 // +kubebuilder:validation:Minimum=0 - LeaseTime int32 `json:"leaseTime"` -} - -// OSIE describes an OSIE to be used with a Hardware. The environment data -// is dependent on the OSIE being used and should be updated with the OSIE reference object. -type OSIE struct { - // OSIERef is a reference to an OSIE object. - OSIERef corev1.LocalObjectReference `json:"osieRef,omitempty"` - + // +kubebuilder:validation:Maximum=4294967295 + LeaseTimeSeconds int64 `json:"leaseTimeSeconds"` } // IPXE describes overrides for IPXE scripts. At least 1 option must be specified. @@ -379,16 +372,18 @@ type WorkflowSpec struct { // If no namespace is specified the Workflow's namespace is assumed. TemplateRef corev1.LocalObjectReference `json:"templateRef,omitempty"` - // TemplateData is user defined data that is injected during template rendering. The data - // structure should be marshalable. + // TemplateParams are a list of key-value pairs that are injected into templates at render + // time. TemplateParams are exposed to templates using a top level .Params key. + // + // For example, if TemplateParams = {"foo": "bar"}, the foo key can be accessed via .Params.foo. // +optional - TemplateData map[string]any `json:"templateData,omitempty"` + TemplateParams map[string]string `json:"templateParams,omitempty"` - // Timeout defines the time the workflow has to complete. The timer begins when the first action - // is requested. When set to 0, no timeout is applied. + // TimeoutSeconds defines the time the workflow has to complete. The timer begins when the first + // action is requested. When set to 0, no timeout is applied. // +kubebuilder:default=0 // +kubebuilder:validation:Minimum=0 - Timeout time.Duration `json:"timeout,omitempty"` + TimeoutSeconds int64 `json:"timeout,omitempty"` } type WorkflowStatus struct { @@ -419,7 +414,8 @@ type ActionStatus struct { // ID uniquely identifies the action status. ID string `json:"id"` - // StartedAt is the time the action was requested. Nil indicates the Action has not started. + // StartedAt is the time the action was started as reported by the client. Nil indicates the + // Action has not started. StartedAt *metav1.Time `json:"startedAt,omitempty"` // LastTransition is the observed time when State transitioned last. @@ -517,24 +513,6 @@ The conditions data structure will only be available on the `Workflow` CRD but i // ConditionType identifies the type of condition. type ConditionType string -// ConditionSeverity expresses the severity of a Condition Type failing. -type ConditionSeverity string - -const ( - // ConditionSeverityError indicates the condition should be treated as an error. - ConditionSeverityError ConditionSeverity = "Error" - - // ConditionSeverityWarning indicates the condition should be investigated but that everything - // may continue to function. - ConditionSeverityWarning ConditionSeverity = "Warning" - - // ConditionSeverityInfo indicates the condition is informational only. - ConditionSeverityInfo ConditionSeverity = "Info" - - // ConditionSeverityNone indicates the condition has no severity. - ConditionSeverityNone ConditionSeverity = "" -) - // ConditionStatus expresses the current state of the condition. type ConditionStatus string @@ -559,14 +537,6 @@ type Condition struct { // Status of the condition. Status ConditionStatus `json:"status"` - // Severity with which to treat failures of this type of condition. - // +kubebuilder:default="Error" - // +optional - Severity ConditionSeverity `json:"severity,omitempty"` - - // LastTransition is the last time the condition transitioned from one status to another. - LastTransition *metav1.Time `json:"lastTransitionTime"` - // Reason is a short CamelCase description for the conditions last transition. // +optional Reason string `json:"reason,omitempty"` @@ -598,6 +568,8 @@ Templates will have access to a subset of `Hardware` data when they are rendered All existing, custom template functions detailed in [`template_funcs.go`](https://github.com/tinkerbell/tink/blob/main/internal/workflow/template_funcs.go) will continue to be available during template rendering. +Additionally, data specified on the new `Workflow.Spec.TemplateParams` field will be available from a top level `.Params` key. This compliments hardware data that is injected via the `.Hardware` key. + ### Hegel API Changes Hegel will undergo a reduction in the endpoints it serves because the data is no longer available on the `Hardware` resource. Minimally, Hegel will serve the following endpoints: