Skip to content

Commit

Permalink
Merge branch 'swoodward/cd-docs' of github.com:pluralsh/documentation…
Browse files Browse the repository at this point in the history
… into swoodward/cd-docs
  • Loading branch information
swoodward90 committed Nov 10, 2023
2 parents 88e5e5d + e312226 commit ea453ac
Show file tree
Hide file tree
Showing 4 changed files with 69 additions and 60 deletions.
6 changes: 4 additions & 2 deletions pages/adding-new-application/background-app-install.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ description: >-

In this guide we will lay out how your user provided values tie in with the deployment process as it relates to the configuration and templating of the app.

When a user sets up a new Plural workspace in a git repository (we'll call that a *deployment repository* going forward) a `workspace.yaml` file is created that stores global values for that cluster such as the cloud account and region, the cluster and VPC name and what subdomain all the applications will be hosted under.
When a user sets up a new Plural workspace in a git repository (we'll call that a _deployment repository_ going forward) a `workspace.yaml` file is created that stores global values for that cluster such as the cloud account and region, the cluster and VPC name and what subdomain all the applications will be hosted under.
Next, the user can install an application bundle using the `plural bundle install <app_name> <bundle>` CLI command.

> Most applications come with more than one bundle, one for each targeted cloud provider.
Expand All @@ -32,6 +32,7 @@ $ tree -L 1 .
```

The `workspace.yaml` would look like this:

```yaml
apiVersion: plural.sh/v1alpha1
kind: ProjectManifest
Expand All @@ -40,7 +41,7 @@ metadata:
spec:
cluster: pluraldev
bucket: pluraldevsh-tf-state
project: "123456765432"
project: '123456765432'
provider: aws
region: us-east-2
owner:
Expand All @@ -53,6 +54,7 @@ spec:
```
And the `context.yaml` like this. In the `spec.configuration` section you can see the recorded user input for each artifact.

```yaml
apiVersion: plural.sh/v1alpha1
kind: Context
Expand Down
2 changes: 1 addition & 1 deletion pages/adding-new-application/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,4 +9,4 @@ description: >-
If you want to contribute your own application to the Plural marketplace we invite you to create and publish your own artifacts.
In this section we guide you through the ins and outs of Plural's application artifacts, how you can build your own with the tools that Plural provides you, and how you can publish them for others to use.

> We are focused on properly compensating any contributions to the Plural platform, which includes a meaningful [bounty](https://github.com/pluralsh/plural-artifacts#plural-contributor-program).
> We are focused on properly compensating any contributions to the Plural platform, which includes a meaningful [bounty](https://github.com/pluralsh/plural-artifacts#plural-contributor-program).
114 changes: 59 additions & 55 deletions pages/adding-new-application/plural-artifact-structure.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@ description: >-
As mentioned in [Background on Application Installations](/adding-new-application/background-app-install),
the Plural CLI creates a wrapper Helm chart and Terraform module for each installed application and inputs the user provided values for that installation.
Some extra configuration files are necessary in the application's artifact for the Plural API to be able to understand:

- the Helm charts and Terraform modules dependencies to run them through its templating engine
- dependencies on other Plural artifacts
- platform specific components and infrastructure configurations
Expand Down Expand Up @@ -85,7 +86,6 @@ $ tree .

Let's disect this artifact's structure.


## Helm

The `helm` directory contains the app's Helm chart as it will be available through the Plural API and used by the Plural CLI to configure and deploy the Kubernetes components into your cluster.
Expand All @@ -105,7 +105,8 @@ Plural leverages dependency tracking of applications to achieve a high degree of
Dependencies between artifacts are defined in the recipe files (see below).
Dependencies are also tracked between the Helm charts and Terraform modules of other applications in a dedicated `deps.yaml` file in each chart's or module's directory.
For example, for Dagster's Helm chart you would list required dependencies on:
- the `bootstrap` application's Helm chart

- the `bootstrap` application's Helm chart
- the `ingress-nginx` application's Helm chart
- the `postrges` operator application's Helm chart

Expand All @@ -120,33 +121,33 @@ metadata:
spec:
breaking: true
dependencies:
- type: helm
name: bootstrap
repo: bootstrap
version: '>= 0.5.1'
- type: helm
name: ingress-nginx
repo: ingress-nginx
version: ">= 0.1.2"
- type: helm
name: postgres
repo: postgres
version: ">= 0.1.6"
- type: terraform
name: aws
repo: dagster
version: '>= 0.1.0'
optional: true
- type: terraform
name: azure
repo: dagster
version: '>= 0.1.0'
optional: true
- type: terraform
name: gcp
repo: dagster
version: '>= 0.1.0'
optional: true
- type: helm
name: bootstrap
repo: bootstrap
version: '>= 0.5.1'
- type: helm
name: ingress-nginx
repo: ingress-nginx
version: '>= 0.1.2'
- type: helm
name: postgres
repo: postgres
version: '>= 0.1.6'
- type: terraform
name: aws
repo: dagster
version: '>= 0.1.0'
optional: true
- type: terraform
name: azure
repo: dagster
version: '>= 0.1.0'
optional: true
- type: terraform
name: gcp
repo: dagster
version: '>= 0.1.0'
optional: true
```
## Terraform
Expand All @@ -157,9 +158,10 @@ Most commonly you'd find object storage alongside their IAM resources, or additi
Sometimes it will also include Kubernetes resources like service accounts, if their deployment cannot be achieved through the artifact's Helm chart in a convenient manner.

> One peculiarity about the Terraform modules is that, at the very least, they need to contain the Kubernetes namespace for your application.
This is because during a `plural deploy` with the Plural CLI the `terraform apply` will always run before the `helm install` step.
> This is because during a `plural deploy` with the Plural CLI the `terraform apply` will always run before the `helm install` step.

Just like the Helm chart, the Terraform modules also contain a `deps.yaml` file that inform the Plural API about dependencies on other modules.

```
apiVersion: plural.sh/v1alpha1
kind: Dependencies
Expand All @@ -176,7 +178,6 @@ spec:
- aws
```
## Plural Files
The `plural` directory contains the artifact's packaging information (`plural/recipes`), metadata (`plural/tags` and `plural/icons`), and application specific instructions (`plural/docs` and `plural/notes.tpl`).
Expand All @@ -185,6 +186,7 @@ On the top-level directory of each artifact you'll also find a`repository.yaml`.
The `repository.yaml` and recipe YAMLs are an integral part of Plural's artifact packaging format.
### `repository.yaml`
```yaml
name: dagster
description: A data orchestration platform for the development, production, and observation of data assets.
Expand All @@ -199,53 +201,55 @@ oauthSettings:
uriFormat: https://{domain}/oauth2/callback
authMethod: POST
tags:
- tag: dag
- tag: data
- tag: data-pipelines
- tag: dag
- tag: data
- tag: data-pipelines
```

The metadata in this file informs the Plural API about the application this artifact envelopes.
It will store its name, category and description, where it can find the icon and docs to display in the marketplace,
the notes template to prompt after installation, as well as links to any upstream git repository or homepage (if applicable).

`oauthSettings` specifies the URI format for the OIDC callback address and informs the Plural API how to setup the OIDC endpoint for your application if you use it.

> Behind the scenes, every `plural bundle install` triggers the OIDC client creation when you answer with `yes` on the OIDC prompt.
This happens, because every client needs to be created before a `plural build` which then inputs the client info into the helm chart.
> This happens, because every client needs to be created before a `plural build` which then inputs the client info into the helm chart.
The `private` flag controls whether the artifact's bundles are published publicly or privately on a `plural push` or `plural apply` (see [Publishing](/adding-new-application/publishing)).

### `plural/recipes/dagster-aws.yaml`

```yaml
name: dagster-aws
description: Installs dagster on an aws eks cluster
provider: AWS
primary: true
dependencies:
- repo: bootstrap
name: aws-k8s
- repo: ingress-nginx
name: ingress-nginx-aws
- repo: postgres
name: aws-postgres
- repo: bootstrap
name: aws-k8s
- repo: ingress-nginx
name: ingress-nginx-aws
- repo: postgres
name: aws-postgres
oidcSettings:
uriFormat: https://{domain}/oauth2/callback
authMethod: POST
domainKey: hostname
sections:
- name: dagster
configuration:
- name: dagsterBucket
type: BUCKET
documentation: s3 bucket for storing dagster logs
default: dagster
- name: hostname
type: DOMAIN
documentation: fqdn on which to deploy your dagster instance
items:
- type: TERRAFORM
name: aws
- type: HELM
name: dagster
- name: dagster
configuration:
- name: dagsterBucket
type: BUCKET
documentation: s3 bucket for storing dagster logs
default: dagster
- name: hostname
type: DOMAIN
documentation: fqdn on which to deploy your dagster instance
items:
- type: TERRAFORM
name: aws
- type: HELM
name: dagster
```
The recipe file ties a bundle together, with one dedicated recipe per cloud provider.
Expand Down
7 changes: 5 additions & 2 deletions pages/adding-new-application/publishing.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,16 +11,17 @@ To publish an artifact you first need to create a publisher with your Plural acc

1. Click on your profile on the bottom left on `app.plural.sh`.
2. Select `create publisher`.
3. Enter your information.
3. Enter your information.
4. Hit save.

## Publish an Artifact with the Plural CLI

When you're done with implementing your artifact and you have created a publisher you're ready to publish it.
When you're done with implementing your artifact and you have created a publisher you're ready to publish it.

1. Navigate to the directory of your artifact in your local copy of the [Plural artifacts git repository](https://github.com/pluralsh/plural-artifacts).
2. Locate the `Pluralfile`.
3. Enter the name of your publisher as the first argument in the `ATTRIBUTES` line. E.g. if your publisher is called `mypublisher`, it should look like this:

```
REPO dagster
ATTRIBUTES mypublisher repository.yaml
Expand All @@ -29,7 +30,9 @@ TF terraform/*
HELM helm/*
RECIPE plural/recipes/*
```

4. Finally, publish your artifact with the command `plural apply -f Pluralfile`. E.g. for our dagster example

```console
$ plural apply -f Pluralfile
Setting attributes for dagster
Expand Down

0 comments on commit ea453ac

Please sign in to comment.