From 8187419ec3d6274cb10cb39315e151fb3c5a4312 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Nils=20Gustav=20Str=C3=A5b=C3=B8?= Date: Thu, 9 Nov 2023 16:40:51 +0100 Subject: [PATCH 01/16] build-deploy documentation --- .../docs/src/guides/build-and-deploy/index.md | 50 ++++++++++++++++++- 1 file changed, 49 insertions(+), 1 deletion(-) diff --git a/public-site/docs/src/guides/build-and-deploy/index.md b/public-site/docs/src/guides/build-and-deploy/index.md index 5d0c79e9..50eeab76 100644 --- a/public-site/docs/src/guides/build-and-deploy/index.md +++ b/public-site/docs/src/guides/build-and-deploy/index.md @@ -4,6 +4,54 @@ title: Build and deploy # Build and deploy +The [`build-deploy`](../../start/workflows/) pipeline builds and deploys Docker files for components and jobs that do not have the [`image`](../../references/reference-radix-config/#image) property set in [`radixconfig.yaml`](../../references/reference-radix-config). The location of the Docker file for each component/job is defined in the [`dockerfileName`](../../references/reference-radix-config/#dockerfilename) and [`src`](../../references/reference-radix-config/#src) properties. + +A `build-deploy` pipeline job can be created manually from [`Radix Web Console`](https://console.radix.equinor.com/) or [`Radix CLI`](../../docs/topic-radix-cli/), or automatically when code is pushed to the application's Github repository if a [GitHub webhook](https://docs.github.com/en/developers/webhooks-and-events/webhooks/about-webhooks) is configured. Instructions on how to register a Github webhook is found in the `Webhook` section on application's configuration page in Radix Web Console. + +When a pipeline job is created from a Github webhook, Radix will compare the commit ID from the webhook request body with the commit ID of the active deployment, to detect which directories have changed. The list of changed directories are then compared to the `src` property for each component and job, and if any directory is equal to, or a child of `src`, the component/job is built. + +To illustrate, consider the following radixconfig.yaml + +``` radixconfig.yaml +apiVersion: radix.equinor.com/v1 +kind: RadixApplication +metadata: + name: myapp +spec: + components: + - name: foo + src: ./foo + - name: bar + src: ./bar + - name: baz + src: . + dockerfileName: baz.Dockerfile +``` + +and directory structure + +``` directory-structure +├── foo/ +│ ├── images/ +│ │ └── logo.jpg +│ ├── foo.js +│ └── Dockerfile +├── bar/ +│ ├── server.js +│ └── Dockerfile +├── baz.Dockerfile +├── README.md +└── radixconfig.yaml +``` + + +If Radix detects that `radixconfig.yaml` or `build secret` values have change, all components and jobs are built. + + + +For manually created pipeline jobs, Radix will always build all components and jobs. + + Radix CI-CD pipeline can build a Radix application from the source code, located in the [GitHub repository](../../start/requirements/#repository) and then deployed to the Radix, using the [Build and deploy](../../start/workflows/) workflow. When a Radix application component need to be built from the source code, it need to have an option [src](../../references/reference-radix-config/#src) or [dockerfileName](../../references/reference-radix-config/#dockerfilename) in the [radixconfig.yaml](../../references/reference-radix-config). @@ -16,7 +64,7 @@ There are few options how the code can be located in the GitHub repository for a * [Multiple components in the root](./example-multiple-components-application-with-source-in-root.md) * [Multiple Radix applications with the source in the same GitHub repository](./example-monorepo-for-multiple-applications-with-same-repository.md) -The Radix provides an option to use a [GitHub webhook](https://docs.github.com/en/developers/webhooks-and-events/webhooks/about-webhooks), registered for a Radix application GitHub repository. If this webhook is used, GitHub sends events to Radix on push of changes to this repository. These events trigger new pipeline jobs with `Build and deploy` workflow for Radix applications. These jobs build and deploy Radix components, which have an option `src` or `dockerfileName` defined in the `radixconfig.yaml`. +Radix provides an option to use a [GitHub webhook](https://docs.github.com/en/developers/webhooks-and-events/webhooks/about-webhooks), registered for a Radix application GitHub repository. If this webhook is used, GitHub sends events to Radix on push of changes to this repository. These events trigger new pipeline jobs with `Build and deploy` workflow for Radix applications. These jobs build and deploy Radix components, which have an option `src` or `dockerfileName` defined in the `radixconfig.yaml`. In addition to the source code and configuration file, there can be other files and folders, which are not part of deployments: `README.md`, `CHANGE_LOG.md` for an application, `.gitignore`, folder with documentation, etc. Changes in these files do not affect components, built by Radix from sources (if these changed files and folders are outside of locations, specified in the component `src` option). To avoid unnecessary re-deployment of a these Radix components, `Build and deploy` pipeline workflow analyses changes, committed to a repository to find out if source files were changed for these components. When such changes are detected, the Radix application is built and deployed, otherwise the pipeline job is stopped with a status `Stopped no changes`. From 58c075eb5d90080e5fc4928b0b9bda58e9a44435 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Nils=20Gustav=20Str=C3=A5b=C3=B8?= Date: Fri, 10 Nov 2023 13:25:37 +0100 Subject: [PATCH 02/16] build-deploy pipeline guide --- .../docs/src/guides/build-and-deploy/index.md | 109 ++++++++++-------- 1 file changed, 64 insertions(+), 45 deletions(-) diff --git a/public-site/docs/src/guides/build-and-deploy/index.md b/public-site/docs/src/guides/build-and-deploy/index.md index 50eeab76..ff59f595 100644 --- a/public-site/docs/src/guides/build-and-deploy/index.md +++ b/public-site/docs/src/guides/build-and-deploy/index.md @@ -8,9 +8,29 @@ The [`build-deploy`](../../start/workflows/) pipeline builds and deploys Docker A `build-deploy` pipeline job can be created manually from [`Radix Web Console`](https://console.radix.equinor.com/) or [`Radix CLI`](../../docs/topic-radix-cli/), or automatically when code is pushed to the application's Github repository if a [GitHub webhook](https://docs.github.com/en/developers/webhooks-and-events/webhooks/about-webhooks) is configured. Instructions on how to register a Github webhook is found in the `Webhook` section on application's configuration page in Radix Web Console. -When a pipeline job is created from a Github webhook, Radix will compare the commit ID from the webhook request body with the commit ID of the active deployment, to detect which directories have changed. The list of changed directories are then compared to the `src` property for each component and job, and if any directory is equal to, or a child of `src`, the component/job is built. +For manually created pipeline jobs, Radix will always build all components and jobs, but when a pipeline job is created from a Github webhook, Radix compares the commit ID from the webhook request body with the commit ID of the active deployment to detect which directories have changed. The list of changed directories are then compared to the `src` property for each component and job, and if any directory is equal to, or a child of `src`, the component/job is built. Radix will use the image from the current active deployment for components and jobs that do not need to be built. -To illustrate, consider the following radixconfig.yaml +When Radix detects that `radixconfig.yaml` or `build secret` values have changed, all components and jobs are built, and if no changes are detected, the pipeline job is stopped with status `Stopped no changes`. + +#### Example + +To illustrate, consider an application with the following directory layout + +``` directory-structure +├── foo/ +│ ├── images/ +│ │ └── logo.jpg +│ ├── main.js +│ └── Dockerfile +├── bar/ +│ ├── main.js +│ ├── README.md +│ └── Dockerfile +├── CHANGE_LOG.md +└── radixconfig.yaml +``` + +and corresponding radixconfig.yaml ``` radixconfig.yaml apiVersion: radix.equinor.com/v1 @@ -18,62 +38,61 @@ kind: RadixApplication metadata: name: myapp spec: + build: + secrets: + - SECRET1 + - SECRET2 components: - name: foo - src: ./foo + src: /foo - name: bar - src: ./bar - - name: baz - src: . - dockerfileName: baz.Dockerfile + src: /bar ``` -and directory structure +Components `foo` and `bar` are configured with `src` set to separate directories. We assume that an active deployment already exist, created from a `build-deploy` pipeline job that built images named `foo:tag1` and `bar:tag1` for the `foo` and `bar` components respectively. + +The application developer performs the following actions: +1. A new file, `/foo/images/header.jpg`, is pushed to the repository. This triggers a new `build-deploy` pipeline job from the Github webhook. Radix compares the commit ID received from the webhook with the commit ID of the current deployment, and detects that directory `/foo/images` has changed. This directory is a child of `/foo` which is defined as `src` for the `foo` component. Radix will therefore build a new image, e.g. `foo:tag2`, for this component. The `bar` component is unchanged, and Radix will reuse the image from the current active deployment. Once the build step is completed, the pipeline job creates a new deployment where `foo` is configured to run the `foo:tag2` image and `bar` is configured to run `bar:tag1`. +1. The `/bar/README.md` file is updated, and a new `build-deploy` job is triggered from the Github webhook. Radix detects that the `/bar` directory has changed, which is the `src` of the `bar` component. Radix will therefore build a new image, e.g. `bar:tag3`, for this component, and reuse the image `foo:tag2` for the unchanged `foo` component. +1. The `/CHANGE_LOG.md` is updated, and a new `build-deploy` pipeline job is triggered. Radix detects that the `/` (root) directory has changed. This directory does is not within the `src` directory structure of any components, and Radix will abort the pipeline job with status `Stopped no changes`. +1. Files `/foo/main.js` and `/bar/main.js` are updated, and a new `build-deploy` pipeline job is triggered. Radix detects that directories `/foo` and `/bar` have changed, and builds new images, `foo:tag4` and `bar:tag4`, for both components. The new deployment is configured to use image `foo:tag4` for `foo` and `bar:tag4` for bar. +1. The `radixconfig.yaml` is updated, and a new `build-deploy` pipeline job is triggered. Radix detects that the `/` (root) directory has changed. Even though this directory does not match the `src` of any components, Radix sees that `radixconfig.yaml` was modified and will therefore build new images for all components. +1. The developer updates the value for build secret `SECRET1`. Later, the `/CHANGE_LOG.md` is updated, and a new `build-deploy` pipeline job is triggered. Radix detects that the `/` (root) directory has changed, which does match the `src` of any components, but since build secrets have changed since last deployment, Radix builds new images, `foo:tag5` and `bar:tag5`, for both components. The new deployment is configured to use image `foo:tag5` for `foo` and `bar:tag5` for bar. + +The build change detection described in the previous example example would not have worked had the Docker files been placed in the same directory, e.g. `/` (root). In the next example, both of the components have `src` set to `.` (root). Any change will always match, or be a child of `src`, and Radix will therefore always build both components. ``` directory-structure ├── foo/ │ ├── images/ │ │ └── logo.jpg -│ ├── foo.js -│ └── Dockerfile +│ └── main.js ├── bar/ -│ ├── server.js -│ └── Dockerfile -├── baz.Dockerfile -├── README.md +│ ├── main.js +│ └── README.md +├── foo.Dockerfile +├── bar.Dockerfile +├── CHANGE_LOG.md └── radixconfig.yaml ``` +``` radixconfig.yaml +apiVersion: radix.equinor.com/v1 +kind: RadixApplication +metadata: + name: myapp +spec: + components: + - name: foo + dockerfileName: foo.Dockerfile + src: . + - name: bar + dockerfileName: bar.Dockerfile + src: . +``` -If Radix detects that `radixconfig.yaml` or `build secret` values have change, all components and jobs are built. - - - -For manually created pipeline jobs, Radix will always build all components and jobs. - - -Radix CI-CD pipeline can build a Radix application from the source code, located in the [GitHub repository](../../start/requirements/#repository) and then deployed to the Radix, using the [Build and deploy](../../start/workflows/) workflow. - -When a Radix application component need to be built from the source code, it need to have an option [src](../../references/reference-radix-config/#src) or [dockerfileName](../../references/reference-radix-config/#dockerfilename) in the [radixconfig.yaml](../../references/reference-radix-config). - -There are few options how the code can be located in the GitHub repository for a Radix application: - -* [One component with the source in a sub-folder](./example-single-component-application-with-source-in-subfolder.md) -* [One component with the source in the root](./example-single-component-application-with-source-in-root.md) -* [Multiple components with the code in sub-folders](./example-multiple-components-application-with-source-in-subfolders.md) -* [Multiple components in the root](./example-multiple-components-application-with-source-in-root.md) -* [Multiple Radix applications with the source in the same GitHub repository](./example-monorepo-for-multiple-applications-with-same-repository.md) - -Radix provides an option to use a [GitHub webhook](https://docs.github.com/en/developers/webhooks-and-events/webhooks/about-webhooks), registered for a Radix application GitHub repository. If this webhook is used, GitHub sends events to Radix on push of changes to this repository. These events trigger new pipeline jobs with `Build and deploy` workflow for Radix applications. These jobs build and deploy Radix components, which have an option `src` or `dockerfileName` defined in the `radixconfig.yaml`. - -In addition to the source code and configuration file, there can be other files and folders, which are not part of deployments: `README.md`, `CHANGE_LOG.md` for an application, `.gitignore`, folder with documentation, etc. Changes in these files do not affect components, built by Radix from sources (if these changed files and folders are outside of locations, specified in the component `src` option). To avoid unnecessary re-deployment of a these Radix components, `Build and deploy` pipeline workflow analyses changes, committed to a repository to find out if source files were changed for these components. When such changes are detected, the Radix application is built and deployed, otherwise the pipeline job is stopped with a status `Stopped no changes`. - -Conditions when a `Build and deploy` pipeline workflow builds and deploys components: - -* When files are changed in the GitHub repository folder, specified for a Radix application component in the `src` option. -* When a component does not have an option `image`, and an option `src` is set as `src: .` (a root of the repository) or it is omitted (which falls down to a default value `src: .`). -* When the configuration file [radixconfig.yaml](../../references/reference-radix-config) (or the one, [specified in the Radix application config form](../monorepo/#custom-radix-configuration-file-name) is changed in the config branch. -* When there is [sub-pipeline (Tekton pipeline)](../sub-pipeline) in the repository branch, which received changes. -* When the pipeline job is triggered manually from the Radix console or by [Radix CLI](https://github.com/equinor/radix-cli) +#### Summary -Another reason to build and deploy Radix application only when its component source code is changed is a [monorepo strategy](../monorepo/) - when multiple applications use the same GitHub repository for their source code, but with different sub-folders of this repository for their components. Changes for components of one application do not re-build and re-deploy another application. +- Pipeline jobs created manually from `Radix Web Console` or `Radix CLI` will always build all components and jobs. +- When pipeline jobs are created from a `Github webhook`, Radix will only build component/jobs that are affected by the changed directories. +- When Radix detects that `radixconfig.yaml` is modified, all components and jobs are built. +- When `build secret` values are updated, the new `build-deploy` will build all components and jobs. From e75c7a7ea42dd88127bd562ebeed120e63186ccc Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Nils=20Gustav=20Str=C3=A5b=C3=B8?= Date: Fri, 10 Nov 2023 13:53:18 +0100 Subject: [PATCH 03/16] misc changes --- .../docs/src/guides/build-and-deploy/index.md | 32 ++++++++++--------- 1 file changed, 17 insertions(+), 15 deletions(-) diff --git a/public-site/docs/src/guides/build-and-deploy/index.md b/public-site/docs/src/guides/build-and-deploy/index.md index ff59f595..f3f8d38f 100644 --- a/public-site/docs/src/guides/build-and-deploy/index.md +++ b/public-site/docs/src/guides/build-and-deploy/index.md @@ -6,15 +6,17 @@ title: Build and deploy The [`build-deploy`](../../start/workflows/) pipeline builds and deploys Docker files for components and jobs that do not have the [`image`](../../references/reference-radix-config/#image) property set in [`radixconfig.yaml`](../../references/reference-radix-config). The location of the Docker file for each component/job is defined in the [`dockerfileName`](../../references/reference-radix-config/#dockerfilename) and [`src`](../../references/reference-radix-config/#src) properties. -A `build-deploy` pipeline job can be created manually from [`Radix Web Console`](https://console.radix.equinor.com/) or [`Radix CLI`](../../docs/topic-radix-cli/), or automatically when code is pushed to the application's Github repository if a [GitHub webhook](https://docs.github.com/en/developers/webhooks-and-events/webhooks/about-webhooks) is configured. Instructions on how to register a Github webhook is found in the `Webhook` section on application's configuration page in Radix Web Console. +A `build-deploy` pipeline job can be created manually from [`Radix Web Console`](https://console.radix.equinor.com/) or [`Radix CLI`](../../docs/topic-radix-cli/), or automatically when code is pushed to the application's Github repository, if a [GitHub webhook](https://docs.github.com/en/developers/webhooks-and-events/webhooks/about-webhooks) is configured. Instructions on how to configure a Github webhook can be found in the `Webhook` section on the application's configuration page in Radix Web Console. -For manually created pipeline jobs, Radix will always build all components and jobs, but when a pipeline job is created from a Github webhook, Radix compares the commit ID from the webhook request body with the commit ID of the active deployment to detect which directories have changed. The list of changed directories are then compared to the `src` property for each component and job, and if any directory is equal to, or a child of `src`, the component/job is built. Radix will use the image from the current active deployment for components and jobs that do not need to be built. +For manually created pipeline jobs, Radix will always build all components and jobs. When a pipeline job is created from a Github webhook, Radix compares the commit ID from the webhook request body with the commit ID of the active deployment to detect which directories have changed. The list of changed directories are then compared to the `src` property for each component and job, and if any of the changed directories are equal to, or a child of `src`, a new image is built for the component/job. Radix will reuse the image from the current active deployment for components and jobs that have not changed. -When Radix detects that `radixconfig.yaml` or `build secret` values have changed, all components and jobs are built, and if no changes are detected, the pipeline job is stopped with status `Stopped no changes`. +When Radix detects that `radixconfig.yaml` or `build secret` values have changed, all components and jobs are built. + +If no changes are detected, the pipeline job is stopped with status `Stopped no changes`. #### Example -To illustrate, consider an application with the following directory layout +To illustrate, consider an application with the following directory layout: ``` directory-structure ├── foo/ @@ -30,7 +32,7 @@ To illustrate, consider an application with the following directory layout └── radixconfig.yaml ``` -and corresponding radixconfig.yaml +and corresponding radixconfig.yaml: ``` radixconfig.yaml apiVersion: radix.equinor.com/v1 @@ -49,17 +51,17 @@ spec: src: /bar ``` -Components `foo` and `bar` are configured with `src` set to separate directories. We assume that an active deployment already exist, created from a `build-deploy` pipeline job that built images named `foo:tag1` and `bar:tag1` for the `foo` and `bar` components respectively. +Components `foo` and `bar` are configured with `src` set to separate directories. We assume that an active deployment already exist, created from a `build-deploy` pipeline that built and deployed images named `foo:tag1` and `bar:tag1` for the `foo` and `bar` components respectively. The application developer performs the following actions: -1. A new file, `/foo/images/header.jpg`, is pushed to the repository. This triggers a new `build-deploy` pipeline job from the Github webhook. Radix compares the commit ID received from the webhook with the commit ID of the current deployment, and detects that directory `/foo/images` has changed. This directory is a child of `/foo` which is defined as `src` for the `foo` component. Radix will therefore build a new image, e.g. `foo:tag2`, for this component. The `bar` component is unchanged, and Radix will reuse the image from the current active deployment. Once the build step is completed, the pipeline job creates a new deployment where `foo` is configured to run the `foo:tag2` image and `bar` is configured to run `bar:tag1`. +1. A new file, `/foo/images/header.jpg`, is pushed to the repository. This triggers a new `build-deploy` pipeline from the Github webhook. Radix compares the commit ID received from the webhook with the commit ID of the current deployment, and detects that directory `/foo/images` has changed. This directory is a child of `/foo`, which is defined as `src` for the `foo` component. Radix will therefore build a new image, e.g. `foo:tag2`, for this component. The `bar` component is unchanged, and Radix will reuse image `bar:tag1` from the current active deployment. Once the build step is completed, the pipeline creates a new deployment where `foo` is configured to run the new `foo:tag2` image and `bar` is configured to run `bar:tag1`. 1. The `/bar/README.md` file is updated, and a new `build-deploy` job is triggered from the Github webhook. Radix detects that the `/bar` directory has changed, which is the `src` of the `bar` component. Radix will therefore build a new image, e.g. `bar:tag3`, for this component, and reuse the image `foo:tag2` for the unchanged `foo` component. -1. The `/CHANGE_LOG.md` is updated, and a new `build-deploy` pipeline job is triggered. Radix detects that the `/` (root) directory has changed. This directory does is not within the `src` directory structure of any components, and Radix will abort the pipeline job with status `Stopped no changes`. -1. Files `/foo/main.js` and `/bar/main.js` are updated, and a new `build-deploy` pipeline job is triggered. Radix detects that directories `/foo` and `/bar` have changed, and builds new images, `foo:tag4` and `bar:tag4`, for both components. The new deployment is configured to use image `foo:tag4` for `foo` and `bar:tag4` for bar. -1. The `radixconfig.yaml` is updated, and a new `build-deploy` pipeline job is triggered. Radix detects that the `/` (root) directory has changed. Even though this directory does not match the `src` of any components, Radix sees that `radixconfig.yaml` was modified and will therefore build new images for all components. -1. The developer updates the value for build secret `SECRET1`. Later, the `/CHANGE_LOG.md` is updated, and a new `build-deploy` pipeline job is triggered. Radix detects that the `/` (root) directory has changed, which does match the `src` of any components, but since build secrets have changed since last deployment, Radix builds new images, `foo:tag5` and `bar:tag5`, for both components. The new deployment is configured to use image `foo:tag5` for `foo` and `bar:tag5` for bar. +1. The `/CHANGE_LOG.md` is updated, and a new `build-deploy` pipeline is triggered. Radix detects that the `/` (root) directory has changed. This directory is not within the `src` directory structure of any components, and Radix will abort the pipeline with status `Stopped no changes`. +1. Files `/foo/main.js` and `/bar/main.js` are updated, and a new `build-deploy` pipeline is triggered. Radix detects that directories `/foo` and `/bar` have changed, and builds new images, `foo:tag4` and `bar:tag4`, for both components. The new deployment is configured to use image `foo:tag4` for `foo` and `bar:tag4` for `bar`. +1. The `radixconfig.yaml` is updated, and a new `build-deploy` pipeline is triggered. Radix detects that the `/` (root) directory has changed. Even though this directory does not match the `src` of any components, Radix detects that `radixconfig.yaml` was modified and will therefore build new images, `foo:tag5` and `bar:tag5`, for both components. The new deployment is configured to use image `foo:tag5` for `foo` and `bar:tag5` for `bar`. +1. The developer updates the value for build secret `SECRET1`. Later, the `/CHANGE_LOG.md` is updated, and a new `build-deploy` pipeline is triggered. Radix detects that the `/` (root) directory has changed, which does match the `src` of any components, but since build secrets have changed since last deployment, Radix builds new images, `foo:tag6` and `bar:tag6`, for both components. The new deployment is configured to use image `foo:tag6` for `foo` and `bar:tag6` for `bar`. -The build change detection described in the previous example example would not have worked had the Docker files been placed in the same directory, e.g. `/` (root). In the next example, both of the components have `src` set to `.` (root). Any change will always match, or be a child of `src`, and Radix will therefore always build both components. +The build change detection described in the previous example example would not have worked had the Docker files been placed in the same directory, e.g. `/` (root). In the next example, both components have `src` set to `.` (root). Any change will always match, or be a child of `src`, and Radix will therefore always build both components. ``` directory-structure ├── foo/ @@ -92,7 +94,7 @@ spec: #### Summary -- Pipeline jobs created manually from `Radix Web Console` or `Radix CLI` will always build all components and jobs. -- When pipeline jobs are created from a `Github webhook`, Radix will only build component/jobs that are affected by the changed directories. +- Pipelines created manually from `Radix Web Console` or `Radix CLI` will always build all components and jobs. +- When pipelines are created from a `Github webhook`, Radix will only build components and jobs that affected by the changes, and reuse images from the active deployment for unchanged components/jobs. - When Radix detects that `radixconfig.yaml` is modified, all components and jobs are built. -- When `build secret` values are updated, the new `build-deploy` will build all components and jobs. +- When `build secret` values are updated, the next pipeline job will build all components and jobs. From c5f3081d1882c4579935f1bb751087824e469074 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Nils=20Gustav=20Str=C3=A5b=C3=B8?= Date: Fri, 10 Nov 2023 13:59:42 +0100 Subject: [PATCH 04/16] show radixconfig as yaml --- public-site/docs/src/guides/build-and-deploy/index.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/public-site/docs/src/guides/build-and-deploy/index.md b/public-site/docs/src/guides/build-and-deploy/index.md index f3f8d38f..50d57f2d 100644 --- a/public-site/docs/src/guides/build-and-deploy/index.md +++ b/public-site/docs/src/guides/build-and-deploy/index.md @@ -34,7 +34,7 @@ To illustrate, consider an application with the following directory layout: and corresponding radixconfig.yaml: -``` radixconfig.yaml +``` yaml apiVersion: radix.equinor.com/v1 kind: RadixApplication metadata: @@ -77,7 +77,7 @@ The build change detection described in the previous example example would not h └── radixconfig.yaml ``` -``` radixconfig.yaml +```yaml apiVersion: radix.equinor.com/v1 kind: RadixApplication metadata: From a648fb273e6482c67c69dcf29d0854308a3447d4 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Nils=20Gustav=20Str=C3=A5b=C3=B8?= Date: Fri, 10 Nov 2023 15:08:10 +0100 Subject: [PATCH 05/16] move summry to top --- public-site/docs/src/guides/build-and-deploy/index.md | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/public-site/docs/src/guides/build-and-deploy/index.md b/public-site/docs/src/guides/build-and-deploy/index.md index 50d57f2d..0a1742e3 100644 --- a/public-site/docs/src/guides/build-and-deploy/index.md +++ b/public-site/docs/src/guides/build-and-deploy/index.md @@ -4,6 +4,14 @@ title: Build and deploy # Build and deploy + +::: tip TL;DR +- Pipeline jobs created manually from `Radix Web Console` or `Radix CLI` will always build all components and jobs. +- For pipeline jobs created from a `Github webhook`, Radix will only build components and jobs that affected by the changes, and reuse images from the active deployment for unchanged components/jobs. +- When Radix detects that `radixconfig.yaml` is modified, all components and jobs are built. +- When values for `build secrets` defined in radixconfig.yaml, are updated, the next pipeline job will build all components and jobs. +::: + The [`build-deploy`](../../start/workflows/) pipeline builds and deploys Docker files for components and jobs that do not have the [`image`](../../references/reference-radix-config/#image) property set in [`radixconfig.yaml`](../../references/reference-radix-config). The location of the Docker file for each component/job is defined in the [`dockerfileName`](../../references/reference-radix-config/#dockerfilename) and [`src`](../../references/reference-radix-config/#src) properties. A `build-deploy` pipeline job can be created manually from [`Radix Web Console`](https://console.radix.equinor.com/) or [`Radix CLI`](../../docs/topic-radix-cli/), or automatically when code is pushed to the application's Github repository, if a [GitHub webhook](https://docs.github.com/en/developers/webhooks-and-events/webhooks/about-webhooks) is configured. Instructions on how to configure a Github webhook can be found in the `Webhook` section on the application's configuration page in Radix Web Console. @@ -98,3 +106,4 @@ spec: - When pipelines are created from a `Github webhook`, Radix will only build components and jobs that affected by the changes, and reuse images from the active deployment for unchanged components/jobs. - When Radix detects that `radixconfig.yaml` is modified, all components and jobs are built. - When `build secret` values are updated, the next pipeline job will build all components and jobs. + From 37b4469ebca0953260cd93951436df21d459faef Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Nils=20Gustav=20Str=C3=A5b=C3=B8?= Date: Fri, 10 Nov 2023 15:08:46 +0100 Subject: [PATCH 06/16] remove summary from bottom --- public-site/docs/src/guides/build-and-deploy/index.md | 8 -------- 1 file changed, 8 deletions(-) diff --git a/public-site/docs/src/guides/build-and-deploy/index.md b/public-site/docs/src/guides/build-and-deploy/index.md index 0a1742e3..0477d4bf 100644 --- a/public-site/docs/src/guides/build-and-deploy/index.md +++ b/public-site/docs/src/guides/build-and-deploy/index.md @@ -99,11 +99,3 @@ spec: dockerfileName: bar.Dockerfile src: . ``` - -#### Summary - -- Pipelines created manually from `Radix Web Console` or `Radix CLI` will always build all components and jobs. -- When pipelines are created from a `Github webhook`, Radix will only build components and jobs that affected by the changes, and reuse images from the active deployment for unchanged components/jobs. -- When Radix detects that `radixconfig.yaml` is modified, all components and jobs are built. -- When `build secret` values are updated, the next pipeline job will build all components and jobs. - From 5c88f34879d8a35a78e20d07f5c2b88188265ced Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Nils=20Gustav=20Str=C3=A5b=C3=B8?= Date: Fri, 10 Nov 2023 15:50:17 +0100 Subject: [PATCH 07/16] add sub-pipeline info --- public-site/docs/src/guides/build-and-deploy/index.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/public-site/docs/src/guides/build-and-deploy/index.md b/public-site/docs/src/guides/build-and-deploy/index.md index 0477d4bf..ab155fcd 100644 --- a/public-site/docs/src/guides/build-and-deploy/index.md +++ b/public-site/docs/src/guides/build-and-deploy/index.md @@ -20,7 +20,7 @@ For manually created pipeline jobs, Radix will always build all components and j When Radix detects that `radixconfig.yaml` or `build secret` values have changed, all components and jobs are built. -If no changes are detected, the pipeline job is stopped with status `Stopped no changes`. +If no changes are detected, and [`sub-pipline`](../sub-pipeline/) is not configured, the pipeline job is stopped with status `Stopped no changes`. #### Example From eef2f62eb45f780318344c2f75838862b25c6820 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Nils=20Gustav=20Str=C3=A5b=C3=B8?= Date: Fri, 10 Nov 2023 16:20:02 +0100 Subject: [PATCH 08/16] misc changes --- .../docs/src/guides/build-and-deploy/index.md | 19 +++++++++++++------ 1 file changed, 13 insertions(+), 6 deletions(-) diff --git a/public-site/docs/src/guides/build-and-deploy/index.md b/public-site/docs/src/guides/build-and-deploy/index.md index ab155fcd..33449cff 100644 --- a/public-site/docs/src/guides/build-and-deploy/index.md +++ b/public-site/docs/src/guides/build-and-deploy/index.md @@ -12,11 +12,11 @@ title: Build and deploy - When values for `build secrets` defined in radixconfig.yaml, are updated, the next pipeline job will build all components and jobs. ::: -The [`build-deploy`](../../start/workflows/) pipeline builds and deploys Docker files for components and jobs that do not have the [`image`](../../references/reference-radix-config/#image) property set in [`radixconfig.yaml`](../../references/reference-radix-config). The location of the Docker file for each component/job is defined in the [`dockerfileName`](../../references/reference-radix-config/#dockerfilename) and [`src`](../../references/reference-radix-config/#src) properties. +The [`build-deploy`](../../start/workflows/) pipeline builds and deploys Docker files for components and jobs that do not have the [`image`](../../references/reference-radix-config/#image) property set in [`radixconfig.yaml`](../../references/reference-radix-config). The location of the Docker file for each component and job is defined in the [`dockerfileName`](../../references/reference-radix-config/#dockerfilename) and [`src`](../../references/reference-radix-config/#src) properties. A `build-deploy` pipeline job can be created manually from [`Radix Web Console`](https://console.radix.equinor.com/) or [`Radix CLI`](../../docs/topic-radix-cli/), or automatically when code is pushed to the application's Github repository, if a [GitHub webhook](https://docs.github.com/en/developers/webhooks-and-events/webhooks/about-webhooks) is configured. Instructions on how to configure a Github webhook can be found in the `Webhook` section on the application's configuration page in Radix Web Console. -For manually created pipeline jobs, Radix will always build all components and jobs. When a pipeline job is created from a Github webhook, Radix compares the commit ID from the webhook request body with the commit ID of the active deployment to detect which directories have changed. The list of changed directories are then compared to the `src` property for each component and job, and if any of the changed directories are equal to, or a child of `src`, a new image is built for the component/job. Radix will reuse the image from the current active deployment for components and jobs that have not changed. +For manually created pipeline jobs, Radix will always build all components and jobs. When a pipeline job is created from a Github webhook, Radix compares the commit ID from the webhook request body with the commit ID of the active deployment to detect which directories have changed. The list of changed directories are then compared to the directory where the Docker file for each component and job is located, and if any of the changed directories are equal to, or a child of the Docker file directory, a new image is built for the component or job. Radix will *reuse* the image from the current active deployment for components and jobs that have not changed. When Radix detects that `radixconfig.yaml` or `build secret` values have changed, all components and jobs are built. @@ -59,17 +59,24 @@ spec: src: /bar ``` -Components `foo` and `bar` are configured with `src` set to separate directories. We assume that an active deployment already exist, created from a `build-deploy` pipeline that built and deployed images named `foo:tag1` and `bar:tag1` for the `foo` and `bar` components respectively. +Components `foo` and `bar` are configured with different directories for their Docker files. We assume that an active deployment already exist, created from a `build-deploy` pipeline that built and deployed images named `foo:tag1` and `bar:tag1` for the `foo` and `bar` components respectively. The application developer performs the following actions: -1. A new file, `/foo/images/header.jpg`, is pushed to the repository. This triggers a new `build-deploy` pipeline from the Github webhook. Radix compares the commit ID received from the webhook with the commit ID of the current deployment, and detects that directory `/foo/images` has changed. This directory is a child of `/foo`, which is defined as `src` for the `foo` component. Radix will therefore build a new image, e.g. `foo:tag2`, for this component. The `bar` component is unchanged, and Radix will reuse image `bar:tag1` from the current active deployment. Once the build step is completed, the pipeline creates a new deployment where `foo` is configured to run the new `foo:tag2` image and `bar` is configured to run `bar:tag1`. -1. The `/bar/README.md` file is updated, and a new `build-deploy` job is triggered from the Github webhook. Radix detects that the `/bar` directory has changed, which is the `src` of the `bar` component. Radix will therefore build a new image, e.g. `bar:tag3`, for this component, and reuse the image `foo:tag2` for the unchanged `foo` component. +- A new file, `/foo/images/header.jpg`, is pushed to the repository: + - A new `build-deploy` pipeline is created by the Github webook. + - Radix compares the commit ID received from the webhook with the commit ID of the current deployment, and detects that directory `/foo/images` has changed. This directory is a child of `/foo`, the directory where the Docker file for component `foo` is located. + - A new image, `foo:tag2`, is built for the `foo` component. + - The `bar` component is unchanged, and Radix will *reuse* image `bar:tag1` from the current active deployment. + - Once the build step is completed, the pipeline creates a new deployment where `foo` is configured to run the new `foo:tag2` image, and `bar` is configured to run the reused `bar:tag1` image. +- The `/bar/README.md` file is updated + - A new `build-deploy` job is created by the Github webhook. + - Radix detects that the `/bar` directory has changed, the is the `src` of the `bar` component. Radix will therefore build a new image, `bar:tag3`, for this component, and *reuse* the image `foo:tag2` for the unchanged `foo` component. 1. The `/CHANGE_LOG.md` is updated, and a new `build-deploy` pipeline is triggered. Radix detects that the `/` (root) directory has changed. This directory is not within the `src` directory structure of any components, and Radix will abort the pipeline with status `Stopped no changes`. 1. Files `/foo/main.js` and `/bar/main.js` are updated, and a new `build-deploy` pipeline is triggered. Radix detects that directories `/foo` and `/bar` have changed, and builds new images, `foo:tag4` and `bar:tag4`, for both components. The new deployment is configured to use image `foo:tag4` for `foo` and `bar:tag4` for `bar`. 1. The `radixconfig.yaml` is updated, and a new `build-deploy` pipeline is triggered. Radix detects that the `/` (root) directory has changed. Even though this directory does not match the `src` of any components, Radix detects that `radixconfig.yaml` was modified and will therefore build new images, `foo:tag5` and `bar:tag5`, for both components. The new deployment is configured to use image `foo:tag5` for `foo` and `bar:tag5` for `bar`. 1. The developer updates the value for build secret `SECRET1`. Later, the `/CHANGE_LOG.md` is updated, and a new `build-deploy` pipeline is triggered. Radix detects that the `/` (root) directory has changed, which does match the `src` of any components, but since build secrets have changed since last deployment, Radix builds new images, `foo:tag6` and `bar:tag6`, for both components. The new deployment is configured to use image `foo:tag6` for `foo` and `bar:tag6` for `bar`. -The build change detection described in the previous example example would not have worked had the Docker files been placed in the same directory, e.g. `/` (root). In the next example, both components have `src` set to `.` (root). Any change will always match, or be a child of `src`, and Radix will therefore always build both components. +The build change detection described in the previous example would not have worked had the Docker files been placed in the same directory, e.g. `/` (root). In the next example, both components have `src` set to `.` (root). Any change will always match, or be a child of `src`, and Radix will therefore always build both components. ``` directory-structure ├── foo/ From 22834fe2cc973c1dd92cbacc18710c0354ed63d2 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Nils=20Gustav=20Str=C3=A5b=C3=B8?= Date: Mon, 13 Nov 2023 10:19:54 +0100 Subject: [PATCH 09/16] misc changes --- .../docs/src/guides/build-and-deploy/index.md | 48 +++++++++++++------ 1 file changed, 33 insertions(+), 15 deletions(-) diff --git a/public-site/docs/src/guides/build-and-deploy/index.md b/public-site/docs/src/guides/build-and-deploy/index.md index 33449cff..b81b4000 100644 --- a/public-site/docs/src/guides/build-and-deploy/index.md +++ b/public-site/docs/src/guides/build-and-deploy/index.md @@ -12,15 +12,15 @@ title: Build and deploy - When values for `build secrets` defined in radixconfig.yaml, are updated, the next pipeline job will build all components and jobs. ::: -The [`build-deploy`](../../start/workflows/) pipeline builds and deploys Docker files for components and jobs that do not have the [`image`](../../references/reference-radix-config/#image) property set in [`radixconfig.yaml`](../../references/reference-radix-config). The location of the Docker file for each component and job is defined in the [`dockerfileName`](../../references/reference-radix-config/#dockerfilename) and [`src`](../../references/reference-radix-config/#src) properties. +The [`build-deploy`](../../start/workflows/) pipeline builds and deploys container images from Docker files for components and jobs that do not have the [`image`](../../references/reference-radix-config/#image) property set in [`radixconfig.yaml`](../../references/reference-radix-config). The location of the Docker file for each component and job is defined in the [`dockerfileName`](../../references/reference-radix-config/#dockerfilename) and [`src`](../../references/reference-radix-config/#src) properties. A `build-deploy` pipeline job can be created manually from [`Radix Web Console`](https://console.radix.equinor.com/) or [`Radix CLI`](../../docs/topic-radix-cli/), or automatically when code is pushed to the application's Github repository, if a [GitHub webhook](https://docs.github.com/en/developers/webhooks-and-events/webhooks/about-webhooks) is configured. Instructions on how to configure a Github webhook can be found in the `Webhook` section on the application's configuration page in Radix Web Console. -For manually created pipeline jobs, Radix will always build all components and jobs. When a pipeline job is created from a Github webhook, Radix compares the commit ID from the webhook request body with the commit ID of the active deployment to detect which directories have changed. The list of changed directories are then compared to the directory where the Docker file for each component and job is located, and if any of the changed directories are equal to, or a child of the Docker file directory, a new image is built for the component or job. Radix will *reuse* the image from the current active deployment for components and jobs that have not changed. +For manually created pipeline jobs, Radix will always build container images for all components and jobs. When a pipeline job is created by a Github webhook, Radix compares the commit ID from the webhook request body with the commit ID of the active deployment, to detect which directories have changed. The list of changed directories are then compared to the path of the Docker file for each component and job. If any of the changed directories are equal to, or a child of the Docker file path, a new image is built for the matching component or job. Radix will reuse the image from the current active deployment for components and jobs that have not changed. When Radix detects that `radixconfig.yaml` or `build secret` values have changed, all components and jobs are built. -If no changes are detected, and [`sub-pipline`](../sub-pipeline/) is not configured, the pipeline job is stopped with status `Stopped no changes`. +If no changes are detected, and [`sub-pipeline`](../sub-pipeline/) is not configured, the pipeline job is stopped with status `Stopped no changes`. #### Example @@ -62,21 +62,39 @@ spec: Components `foo` and `bar` are configured with different directories for their Docker files. We assume that an active deployment already exist, created from a `build-deploy` pipeline that built and deployed images named `foo:tag1` and `bar:tag1` for the `foo` and `bar` components respectively. The application developer performs the following actions: -- A new file, `/foo/images/header.jpg`, is pushed to the repository: +1. Pushes a new file, `/foo/images/header.jpg`, to the repository: - A new `build-deploy` pipeline is created by the Github webook. - - Radix compares the commit ID received from the webhook with the commit ID of the current deployment, and detects that directory `/foo/images` has changed. This directory is a child of `/foo`, the directory where the Docker file for component `foo` is located. + - Radix compares the commit ID received from the webhook with the commit ID of the current deployment, and detects that directory `/foo/images` has changed. This directory is a child of `/foo`, the path to the Docker file for component `foo`. - A new image, `foo:tag2`, is built for the `foo` component. - - The `bar` component is unchanged, and Radix will *reuse* image `bar:tag1` from the current active deployment. - - Once the build step is completed, the pipeline creates a new deployment where `foo` is configured to run the new `foo:tag2` image, and `bar` is configured to run the reused `bar:tag1` image. -- The `/bar/README.md` file is updated + - The `bar` component is unchanged, and Radix will reuse image `bar:tag1` from the current active deployment. + - Once the build step is completed, the pipeline creates a new deployment where `foo` is configured to run the newly built `foo:tag2` image, and `bar` is configured to run the `bar:tag1` image from the previous deployment. +1. The `/bar/README.md` file is updated. - A new `build-deploy` job is created by the Github webhook. - - Radix detects that the `/bar` directory has changed, the is the `src` of the `bar` component. Radix will therefore build a new image, `bar:tag3`, for this component, and *reuse* the image `foo:tag2` for the unchanged `foo` component. -1. The `/CHANGE_LOG.md` is updated, and a new `build-deploy` pipeline is triggered. Radix detects that the `/` (root) directory has changed. This directory is not within the `src` directory structure of any components, and Radix will abort the pipeline with status `Stopped no changes`. -1. Files `/foo/main.js` and `/bar/main.js` are updated, and a new `build-deploy` pipeline is triggered. Radix detects that directories `/foo` and `/bar` have changed, and builds new images, `foo:tag4` and `bar:tag4`, for both components. The new deployment is configured to use image `foo:tag4` for `foo` and `bar:tag4` for `bar`. -1. The `radixconfig.yaml` is updated, and a new `build-deploy` pipeline is triggered. Radix detects that the `/` (root) directory has changed. Even though this directory does not match the `src` of any components, Radix detects that `radixconfig.yaml` was modified and will therefore build new images, `foo:tag5` and `bar:tag5`, for both components. The new deployment is configured to use image `foo:tag5` for `foo` and `bar:tag5` for `bar`. -1. The developer updates the value for build secret `SECRET1`. Later, the `/CHANGE_LOG.md` is updated, and a new `build-deploy` pipeline is triggered. Radix detects that the `/` (root) directory has changed, which does match the `src` of any components, but since build secrets have changed since last deployment, Radix builds new images, `foo:tag6` and `bar:tag6`, for both components. The new deployment is configured to use image `foo:tag6` for `foo` and `bar:tag6` for `bar`. - -The build change detection described in the previous example would not have worked had the Docker files been placed in the same directory, e.g. `/` (root). In the next example, both components have `src` set to `.` (root). Any change will always match, or be a child of `src`, and Radix will therefore always build both components. + - Radix detects that the `/bar` directory has changed, the path to the Docker file for component `bar`. + - A new image, `bar:tag3`, is built for the `bar` component. + - The `foo` component is unchanged, and Radix will reuse image `foo:tag2`. + - The new deployment is configured to run image `foo:tag2` for the `foo` component, and `bar:tag3` for the `bar` component. +1. The `/CHANGE_LOG.md` is updated. + - A new `build-deploy` pipeline is triggered. + - Radix detects that the `/` (root) directory has changed. This directory is not equal to, or a child of the path to the Docker files for any components. + - Radix aborts the pipeline with status `Stopped no changes`. +1. Files `/foo/main.js` and `/bar/main.js` are updated. + - A new `build-deploy` pipeline is triggered. + - Radix detects that directories `/foo` and `/bar` have changed, matching the path to the Docker files for both components. + - New images, `foo:tag4` and `bar:tag4`, are built for the components. + - The new deployment is configured to run image `foo:tag4` for the `foo` component, and `bar:tag4` for the `bar` component. +1. The `radixconfig.yaml` is updated + - A new `build-deploy` pipeline is triggered. + - Radix detects that the `/` (root) directory has changed. This directory does not match the path to the Docker files for any components, but Radix detects that `radixconfig.yaml` is modified. + - New images, `foo:tag5` and `bar:tag5`, are built for the components. + - The new deployment is configured to run image `foo:tag5` for the `foo` component, and `bar:tag5` for the `bar` component. +1. The developer updates the value for build secret `SECRET1`. + - At a later time, the `/CHANGE_LOG.md` is updated, and a new `build-deploy` pipeline is triggered. + - Radix detects that the `/` (root) directory has changed, This directory does not match the path to the Docker files for any components, but Radix detects that `build secrets` have changed since last deployment. + - New images, `foo:tag6` and `bar:tag6`, are built for the components. + - The new deployment is configured to run image `foo:tag6` for the `foo` component, and `bar:tag6` for the `bar` component. + +The build change detection described in the previous example would not have worked had the Docker files been placed in the same directory, e.g. `/` (root). In the next example, the Docker file path for both components is `/` (root). Any change will always match, or be a child of `/`, and Radix will therefore always build both components. ``` directory-structure ├── foo/ From 6ebfb248e6102fcefc28ec601bb1829cf77b0a28 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Nils=20Gustav=20Str=C3=A5b=C3=B8?= Date: Mon, 13 Nov 2023 10:25:28 +0100 Subject: [PATCH 10/16] use Dockerfile instead of Docker file --- .../docs/src/guides/build-and-deploy/index.md | 20 +++++++++---------- 1 file changed, 10 insertions(+), 10 deletions(-) diff --git a/public-site/docs/src/guides/build-and-deploy/index.md b/public-site/docs/src/guides/build-and-deploy/index.md index b81b4000..13c71cc7 100644 --- a/public-site/docs/src/guides/build-and-deploy/index.md +++ b/public-site/docs/src/guides/build-and-deploy/index.md @@ -12,11 +12,11 @@ title: Build and deploy - When values for `build secrets` defined in radixconfig.yaml, are updated, the next pipeline job will build all components and jobs. ::: -The [`build-deploy`](../../start/workflows/) pipeline builds and deploys container images from Docker files for components and jobs that do not have the [`image`](../../references/reference-radix-config/#image) property set in [`radixconfig.yaml`](../../references/reference-radix-config). The location of the Docker file for each component and job is defined in the [`dockerfileName`](../../references/reference-radix-config/#dockerfilename) and [`src`](../../references/reference-radix-config/#src) properties. +The [`build-deploy`](../../start/workflows/) pipeline builds and deploys container images from Dockerfiles for components and jobs that do not have the [`image`](../../references/reference-radix-config/#image) property set in [`radixconfig.yaml`](../../references/reference-radix-config). The location and name of the Dockerfile for each component and job is defined in the [`dockerfileName`](../../references/reference-radix-config/#dockerfilename) and [`src`](../../references/reference-radix-config/#src) properties. A `build-deploy` pipeline job can be created manually from [`Radix Web Console`](https://console.radix.equinor.com/) or [`Radix CLI`](../../docs/topic-radix-cli/), or automatically when code is pushed to the application's Github repository, if a [GitHub webhook](https://docs.github.com/en/developers/webhooks-and-events/webhooks/about-webhooks) is configured. Instructions on how to configure a Github webhook can be found in the `Webhook` section on the application's configuration page in Radix Web Console. -For manually created pipeline jobs, Radix will always build container images for all components and jobs. When a pipeline job is created by a Github webhook, Radix compares the commit ID from the webhook request body with the commit ID of the active deployment, to detect which directories have changed. The list of changed directories are then compared to the path of the Docker file for each component and job. If any of the changed directories are equal to, or a child of the Docker file path, a new image is built for the matching component or job. Radix will reuse the image from the current active deployment for components and jobs that have not changed. +For manually created pipeline jobs, Radix will always build container images for all components and jobs. When a pipeline job is created by a Github webhook, Radix compares the commit ID from the webhook request body with the commit ID of the active deployment, to detect which directories have changed. The list of changed directories are then compared to the path of the Dockerfile for each component and job. If any of the changed directories are equal to, or a child of the Dockerfile path, a new image is built for the matching component or job. Radix will reuse the image from the current active deployment for components and jobs that have not changed. When Radix detects that `radixconfig.yaml` or `build secret` values have changed, all components and jobs are built. @@ -59,42 +59,42 @@ spec: src: /bar ``` -Components `foo` and `bar` are configured with different directories for their Docker files. We assume that an active deployment already exist, created from a `build-deploy` pipeline that built and deployed images named `foo:tag1` and `bar:tag1` for the `foo` and `bar` components respectively. +Components `foo` and `bar` are configured with different directories for their Dockerfiles. We assume that an active deployment already exist, created from a `build-deploy` pipeline that built and deployed images named `foo:tag1` and `bar:tag1` for the `foo` and `bar` components respectively. The application developer performs the following actions: 1. Pushes a new file, `/foo/images/header.jpg`, to the repository: - A new `build-deploy` pipeline is created by the Github webook. - - Radix compares the commit ID received from the webhook with the commit ID of the current deployment, and detects that directory `/foo/images` has changed. This directory is a child of `/foo`, the path to the Docker file for component `foo`. + - Radix compares the commit ID received from the webhook with the commit ID of the current deployment, and detects that directory `/foo/images` has changed. This directory is a child of `/foo`, the path to the Dockerfile for component `foo`. - A new image, `foo:tag2`, is built for the `foo` component. - The `bar` component is unchanged, and Radix will reuse image `bar:tag1` from the current active deployment. - Once the build step is completed, the pipeline creates a new deployment where `foo` is configured to run the newly built `foo:tag2` image, and `bar` is configured to run the `bar:tag1` image from the previous deployment. 1. The `/bar/README.md` file is updated. - A new `build-deploy` job is created by the Github webhook. - - Radix detects that the `/bar` directory has changed, the path to the Docker file for component `bar`. + - Radix detects that the `/bar` directory has changed, the path to the Dockerfile for component `bar`. - A new image, `bar:tag3`, is built for the `bar` component. - The `foo` component is unchanged, and Radix will reuse image `foo:tag2`. - The new deployment is configured to run image `foo:tag2` for the `foo` component, and `bar:tag3` for the `bar` component. 1. The `/CHANGE_LOG.md` is updated. - A new `build-deploy` pipeline is triggered. - - Radix detects that the `/` (root) directory has changed. This directory is not equal to, or a child of the path to the Docker files for any components. + - Radix detects that the `/` (root) directory has changed. This directory is not equal to, or a child of the path to the Dockerfiles for any components. - Radix aborts the pipeline with status `Stopped no changes`. 1. Files `/foo/main.js` and `/bar/main.js` are updated. - A new `build-deploy` pipeline is triggered. - - Radix detects that directories `/foo` and `/bar` have changed, matching the path to the Docker files for both components. + - Radix detects that directories `/foo` and `/bar` have changed, matching the path to the Dockerfiles for both components. - New images, `foo:tag4` and `bar:tag4`, are built for the components. - The new deployment is configured to run image `foo:tag4` for the `foo` component, and `bar:tag4` for the `bar` component. 1. The `radixconfig.yaml` is updated - A new `build-deploy` pipeline is triggered. - - Radix detects that the `/` (root) directory has changed. This directory does not match the path to the Docker files for any components, but Radix detects that `radixconfig.yaml` is modified. + - Radix detects that the `/` (root) directory has changed. This directory does not match the path to the Dockerfiles for any components, but Radix detects that `radixconfig.yaml` is modified. - New images, `foo:tag5` and `bar:tag5`, are built for the components. - The new deployment is configured to run image `foo:tag5` for the `foo` component, and `bar:tag5` for the `bar` component. 1. The developer updates the value for build secret `SECRET1`. - At a later time, the `/CHANGE_LOG.md` is updated, and a new `build-deploy` pipeline is triggered. - - Radix detects that the `/` (root) directory has changed, This directory does not match the path to the Docker files for any components, but Radix detects that `build secrets` have changed since last deployment. + - Radix detects that the `/` (root) directory has changed, This directory does not match the path to the Dockerfiles for any components, but Radix detects that `build secrets` have changed since last deployment. - New images, `foo:tag6` and `bar:tag6`, are built for the components. - The new deployment is configured to run image `foo:tag6` for the `foo` component, and `bar:tag6` for the `bar` component. -The build change detection described in the previous example would not have worked had the Docker files been placed in the same directory, e.g. `/` (root). In the next example, the Docker file path for both components is `/` (root). Any change will always match, or be a child of `/`, and Radix will therefore always build both components. +The build change detection described in the previous example would not have worked had the Dockerfiles been placed in the same directory, e.g. `/` (root). In the next example, the Dockerfile path for both components is `/` (root). Any change will always match, or be a child of `/`, and Radix will therefore always build both components. ``` directory-structure ├── foo/ From be82e9176457cbaf90c4f72892b2ee4a750cdecb Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Nils=20Gustav=20Str=C3=A5b=C3=B8?= Date: Mon, 13 Nov 2023 10:26:53 +0100 Subject: [PATCH 11/16] minor fix --- public-site/docs/src/guides/build-and-deploy/index.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/public-site/docs/src/guides/build-and-deploy/index.md b/public-site/docs/src/guides/build-and-deploy/index.md index 13c71cc7..67e9a8f9 100644 --- a/public-site/docs/src/guides/build-and-deploy/index.md +++ b/public-site/docs/src/guides/build-and-deploy/index.md @@ -12,7 +12,7 @@ title: Build and deploy - When values for `build secrets` defined in radixconfig.yaml, are updated, the next pipeline job will build all components and jobs. ::: -The [`build-deploy`](../../start/workflows/) pipeline builds and deploys container images from Dockerfiles for components and jobs that do not have the [`image`](../../references/reference-radix-config/#image) property set in [`radixconfig.yaml`](../../references/reference-radix-config). The location and name of the Dockerfile for each component and job is defined in the [`dockerfileName`](../../references/reference-radix-config/#dockerfilename) and [`src`](../../references/reference-radix-config/#src) properties. +The [`build-deploy`](../../start/workflows/) pipeline builds and deploys container images from Dockerfiles for components and jobs that do not have the [`image`](../../references/reference-radix-config/#image) property set in [`radixconfig.yaml`](../../references/reference-radix-config). The name and path to the Dockerfile for each component and job is defined in the [`dockerfileName`](../../references/reference-radix-config/#dockerfilename) and [`src`](../../references/reference-radix-config/#src) properties. A `build-deploy` pipeline job can be created manually from [`Radix Web Console`](https://console.radix.equinor.com/) or [`Radix CLI`](../../docs/topic-radix-cli/), or automatically when code is pushed to the application's Github repository, if a [GitHub webhook](https://docs.github.com/en/developers/webhooks-and-events/webhooks/about-webhooks) is configured. Instructions on how to configure a Github webhook can be found in the `Webhook` section on the application's configuration page in Radix Web Console. From fa774fb873e52609bc1b1a83191cdafffc8f1a2a Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Nils=20Gustav=20Str=C3=A5b=C3=B8?= Date: Mon, 13 Nov 2023 10:27:17 +0100 Subject: [PATCH 12/16] minor fix --- public-site/docs/src/guides/build-and-deploy/index.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/public-site/docs/src/guides/build-and-deploy/index.md b/public-site/docs/src/guides/build-and-deploy/index.md index 67e9a8f9..34ff2f54 100644 --- a/public-site/docs/src/guides/build-and-deploy/index.md +++ b/public-site/docs/src/guides/build-and-deploy/index.md @@ -12,7 +12,7 @@ title: Build and deploy - When values for `build secrets` defined in radixconfig.yaml, are updated, the next pipeline job will build all components and jobs. ::: -The [`build-deploy`](../../start/workflows/) pipeline builds and deploys container images from Dockerfiles for components and jobs that do not have the [`image`](../../references/reference-radix-config/#image) property set in [`radixconfig.yaml`](../../references/reference-radix-config). The name and path to the Dockerfile for each component and job is defined in the [`dockerfileName`](../../references/reference-radix-config/#dockerfilename) and [`src`](../../references/reference-radix-config/#src) properties. +The [`build-deploy`](../../start/workflows/) pipeline builds and deploys container images from Dockerfiles for components and jobs that do not have the [`image`](../../references/reference-radix-config/#image) property set in [`radixconfig.yaml`](../../references/reference-radix-config). The name and path of the Dockerfile for each component and job is defined in the [`dockerfileName`](../../references/reference-radix-config/#dockerfilename) and [`src`](../../references/reference-radix-config/#src) properties. A `build-deploy` pipeline job can be created manually from [`Radix Web Console`](https://console.radix.equinor.com/) or [`Radix CLI`](../../docs/topic-radix-cli/), or automatically when code is pushed to the application's Github repository, if a [GitHub webhook](https://docs.github.com/en/developers/webhooks-and-events/webhooks/about-webhooks) is configured. Instructions on how to configure a Github webhook can be found in the `Webhook` section on the application's configuration page in Radix Web Console. From d9c066a1cca4f65dfb9c5fb389f632ee655376ad Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Nils=20Gustav=20Str=C3=A5b=C3=B8?= Date: Mon, 13 Nov 2023 12:44:12 +0100 Subject: [PATCH 13/16] details about src and dockerfileName --- .../docs/src/guides/build-and-deploy/index.md | 14 +++++++++++++ .../reference-radix-config/index.md | 21 +++++++++++++++---- 2 files changed, 31 insertions(+), 4 deletions(-) diff --git a/public-site/docs/src/guides/build-and-deploy/index.md b/public-site/docs/src/guides/build-and-deploy/index.md index 34ff2f54..44df4c12 100644 --- a/public-site/docs/src/guides/build-and-deploy/index.md +++ b/public-site/docs/src/guides/build-and-deploy/index.md @@ -18,6 +18,12 @@ A `build-deploy` pipeline job can be created manually from [`Radix Web Console`] For manually created pipeline jobs, Radix will always build container images for all components and jobs. When a pipeline job is created by a Github webhook, Radix compares the commit ID from the webhook request body with the commit ID of the active deployment, to detect which directories have changed. The list of changed directories are then compared to the path of the Dockerfile for each component and job. If any of the changed directories are equal to, or a child of the Dockerfile path, a new image is built for the matching component or job. Radix will reuse the image from the current active deployment for components and jobs that have not changed. +::: tip +The path to the Dockerfile is defined by the `src` and `dockerfileName` properties. `dockerfileName` can contain path elements relative to `src`, for example `../Dockerfile` or `myfolder/Dockerfile`. + +See [`dockerfileName`](../../references/reference-radix-config/#dockerfilename) and [`src`](../../references/reference-radix-config/#src) for more information. +::: + When Radix detects that `radixconfig.yaml` or `build secret` values have changed, all components and jobs are built. If no changes are detected, and [`sub-pipeline`](../sub-pipeline/) is not configured, the pipeline job is stopped with status `Stopped no changes`. @@ -124,3 +130,11 @@ spec: dockerfileName: bar.Dockerfile src: . ``` + +#### More examples + +* [One component with the source in a sub-folder](./example-single-component-application-with-source-in-subfolder.md) +* [One component with the source in the root](./example-single-component-application-with-source-in-root.md) +* [Multiple components with the code in sub-folders](./example-multiple-components-application-with-source-in-subfolders.md) +* [Multiple components in the root](./example-multiple-components-application-with-source-in-root.md) +* [Multiple Radix applications with the source in the same GitHub repository](./example-monorepo-for-multiple-applications-with-same-repository.md) \ No newline at end of file diff --git a/public-site/docs/src/references/reference-radix-config/index.md b/public-site/docs/src/references/reference-radix-config/index.md index 4c32255f..49813636 100644 --- a/public-site/docs/src/references/reference-radix-config/index.md +++ b/public-site/docs/src/references/reference-radix-config/index.md @@ -187,7 +187,7 @@ spec: port: 5000 ``` -`src` a folder, relative to the repository root, where the component source code and its `Dockerfile` are located to be built within the [Build and deploy](../../guides/build-and-deploy/) workflow of the Radix CI-CD pipeline. By default `src` is `.` - a root of the GitHub repository. +`src` a folder, relative to the repository root, where the `Dockerfile` for the component is located. The Dockerfile is used by the [Build and deploy](../../guides/build-and-deploy/) workflow of the Radix CI-CD pipeline to build a container image for the component. By default `src` is `.` - a root of the GitHub repository. > When the `image` option is set - `src` option is ignored. @@ -197,18 +197,31 @@ spec: spec: components: - name: frontend - dockerfileName: frontend.Dockerfile + dockerfileName: Dockerfile # Absolute path from repository root: /Dockerfile ports: - name: http port: 8080 - name: backend - dockerfileName: backend.Dockerfile + src: . + dockerfileName: backend/Dockerfile # Absolute path from repository root: /backend/Dockerfile ports: - name: http port: 5000 + - name: api + src: api + dockerfileName: "../Dockerfile" # Absolute path from repository root: /Dockerfile + ports: + - name: http + port: 5000 + - name: web + src: web + dockerfileName: "subfolder/Dockerfile" # Absolute path from repository root: /web/subfolder/Dockerfile + ports: + - name: http + port: 5000 ``` -By default, Radix pipeline expects a docker file with a name `Dockefile` in the component source folder, specified in the option `src`. If this file name needs to be different, it can be specified in the option `dockerfileName`. +By default, Radix pipeline expects a docker file with a name `Dockefile` in the component `src` folder. If this file name needs to be different, it can be specified in the option `dockerfileName`. The name can also contain a path relative to `src`. See configuration examples above. > When the `image` option is set - `dockerfileName` option is ignored. From b8750837e3fb4690f6a4c739757f329c6fec7f23 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Nils=20Gustav=20Str=C3=A5b=C3=B8?= Date: Mon, 13 Nov 2023 13:59:51 +0100 Subject: [PATCH 14/16] update --- .../docs/src/guides/build-and-deploy/index.md | 27 ++++++++++--------- 1 file changed, 14 insertions(+), 13 deletions(-) diff --git a/public-site/docs/src/guides/build-and-deploy/index.md b/public-site/docs/src/guides/build-and-deploy/index.md index 44df4c12..cea0d18d 100644 --- a/public-site/docs/src/guides/build-and-deploy/index.md +++ b/public-site/docs/src/guides/build-and-deploy/index.md @@ -89,7 +89,7 @@ The application developer performs the following actions: - Radix detects that directories `/foo` and `/bar` have changed, matching the path to the Dockerfiles for both components. - New images, `foo:tag4` and `bar:tag4`, are built for the components. - The new deployment is configured to run image `foo:tag4` for the `foo` component, and `bar:tag4` for the `bar` component. -1. The `radixconfig.yaml` is updated +1. The `radixconfig.yaml` is updated. - A new `build-deploy` pipeline is triggered. - Radix detects that the `/` (root) directory has changed. This directory does not match the path to the Dockerfiles for any components, but Radix detects that `radixconfig.yaml` is modified. - New images, `foo:tag5` and `bar:tag5`, are built for the components. @@ -100,18 +100,19 @@ The application developer performs the following actions: - New images, `foo:tag6` and `bar:tag6`, are built for the components. - The new deployment is configured to run image `foo:tag6` for the `foo` component, and `bar:tag6` for the `bar` component. -The build change detection described in the previous example would not have worked had the Dockerfiles been placed in the same directory, e.g. `/` (root). In the next example, the Dockerfile path for both components is `/` (root). Any change will always match, or be a child of `/`, and Radix will therefore always build both components. +If the Dockerfiles for the two components in the previous example are placed in the same directory, e.g. `/src`, then Radix will not be able to distinguish between them. Any change in `/src`, or any of its sub-folders, will always match both components. See example below: ``` directory-structure -├── foo/ -│ ├── images/ -│ │ └── logo.jpg -│ └── main.js -├── bar/ -│ ├── main.js -│ └── README.md -├── foo.Dockerfile -├── bar.Dockerfile +├── src/ +│ ├── foo/ +│ │ ├── images/ +│ │ │ └── logo.jpg +│ │ └── main.js +│ ├── bar/ +│ │ ├── main.js +│ │ └── README.md +│ ├── foo.Dockerfile +│ └── bar.Dockerfile ├── CHANGE_LOG.md └── radixconfig.yaml ``` @@ -124,11 +125,11 @@ metadata: spec: components: - name: foo + src: src dockerfileName: foo.Dockerfile - src: . - name: bar + src: src dockerfileName: bar.Dockerfile - src: . ``` #### More examples From f3df7dad73b260e0beb5b07949c7ce75ba94dbcb Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Nils=20Gustav=20Str=C3=A5b=C3=B8?= Date: Mon, 13 Nov 2023 15:57:07 +0100 Subject: [PATCH 15/16] add orchestrating pipeline step log examples --- .../docs/src/guides/build-and-deploy/index.md | 37 +++++++++++++++++++ 1 file changed, 37 insertions(+) diff --git a/public-site/docs/src/guides/build-and-deploy/index.md b/public-site/docs/src/guides/build-and-deploy/index.md index cea0d18d..6fea164e 100644 --- a/public-site/docs/src/guides/build-and-deploy/index.md +++ b/public-site/docs/src/guides/build-and-deploy/index.md @@ -28,6 +28,43 @@ When Radix detects that `radixconfig.yaml` or `build secret` values have changed If no changes are detected, and [`sub-pipeline`](../sub-pipeline/) is not configured, the pipeline job is stopped with status `Stopped no changes`. +The log from the `Orchestrating pipeline` step prints decisions made by Radix whether to build new images, reuse images from current deployment or use images from `image` property in `radixconfig`. + +::: details Log examples + +Component `server` was changed, and a new container image is built. `compute` and `compute2` is unchanged, and images from active deployment is used. Image for `redis` are configured in `image` property in `radixconfig`: +``` +time="2023-11-13T14:44:31Z" level=info msg="Component image source in environments:" +time="2023-11-13T14:44:31Z" level=info msg=" qa:" +time="2023-11-13T14:44:31Z" level=info msg=" - server from build" +time="2023-11-13T14:44:31Z" level=info msg=" - redis from image in radixconfig" +time="2023-11-13T14:44:31Z" level=info msg=" - compute from active deployment" +time="2023-11-13T14:44:31Z" level=info msg=" - compute2 from active deployment" +``` + +Changed `radixconfig`, requiring all components to be built: +``` +time="2023-11-13T14:42:56Z" level=info msg="RadixApplication updated since last deployment to environment qa" +time="2023-11-13T14:42:56Z" level=info msg="Component image source in environments:" +time="2023-11-13T14:42:56Z" level=info msg=" qa:" +time="2023-11-13T14:42:56Z" level=info msg=" - server from build" +time="2023-11-13T14:42:56Z" level=info msg=" - redis from image in radixconfig" +time="2023-11-13T14:42:56Z" level=info msg=" - compute from build" +time="2023-11-13T14:42:56Z" level=info msg=" - compute2 from build" +``` + +Changed `build secret` values, requiring all components to be built: +``` +time="2023-11-13T14:37:44Z" level=info msg="Build secrets updated since last deployment to environment dev" +time="2023-11-13T14:37:44Z" level=info msg="Component image source in environments:" +time="2023-11-13T14:37:44Z" level=info msg=" qa:" +time="2023-11-13T14:37:44Z" level=info msg=" - server from build" +time="2023-11-13T14:37:44Z" level=info msg=" - redis from image in radixconfig" +time="2023-11-13T14:37:44Z" level=info msg=" - compute from build" +time="2023-11-13T14:37:44Z" level=info msg=" - compute2 from build" +``` +::: + #### Example To illustrate, consider an application with the following directory layout: From 9bf4f15dafa2ce98705de3a30a65bf1bb2e698a1 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Nils=20Gustav=20Str=C3=A5b=C3=B8?= Date: Mon, 13 Nov 2023 16:00:10 +0100 Subject: [PATCH 16/16] fix --- public-site/docs/src/guides/build-and-deploy/index.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/public-site/docs/src/guides/build-and-deploy/index.md b/public-site/docs/src/guides/build-and-deploy/index.md index 6fea164e..5dd8be5e 100644 --- a/public-site/docs/src/guides/build-and-deploy/index.md +++ b/public-site/docs/src/guides/build-and-deploy/index.md @@ -32,7 +32,7 @@ The log from the `Orchestrating pipeline` step prints decisions made by Radix wh ::: details Log examples -Component `server` was changed, and a new container image is built. `compute` and `compute2` is unchanged, and images from active deployment is used. Image for `redis` are configured in `image` property in `radixconfig`: +Component `server` was changed, and a new container image is built. `compute` and `compute2` are unchanged, and images from active deployment are used. Image for `redis` is configured in `image` property in `radixconfig`: ``` time="2023-11-13T14:44:31Z" level=info msg="Component image source in environments:" time="2023-11-13T14:44:31Z" level=info msg=" qa:"