Skip to content

Commit

Permalink
Add documentation for new YAML attributes
Browse files Browse the repository at this point in the history
[skip ci]
  • Loading branch information
ivotron committed Jun 9, 2020
1 parent 3b03df8 commit aa62b01
Showing 1 changed file with 9 additions and 24 deletions.
33 changes: 9 additions & 24 deletions docs/sections/cn_workflows.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,21 +4,11 @@ This section introduces the YAML syntax used by Popper, describes the
workflow execution runtime and shows how to execute workflows in
alternative container engines.

> _**NOTE**: Popper also supports the [now-deprecated HCL
> syntax][drophcl] that was introduced in the alpha version of [Github
> Action Workflows][parser]. We strongly recommend the use of Popper's
> own YAML syntax._
[parser]: https://github.blog/2019-02-07-an-open-source-parser-for-github-actions/
[drophcl]: https://github.blog/changelog/2019-09-17-github-actions-will-stop-running-workflows-written-in-hcl/

## Syntax

A Popper workflow file looks like the following:

```yaml
version: '1'

steps:
- uses: docker://alpine:3.9
args: ["ls", "-la"]
Expand All @@ -44,15 +34,15 @@ options that are applied to the workflow.
The following table describes the attributes that can be used for a
step. All attributes are optional with the exception of the `uses` attribute.

| Attribute | Description |
| :--------- | :-------------------- |
| `uses` | The Docker image that will be executed for that step. For example,<br>`uses: docker://node:10`. See **"Referencing images in a step"** section below for more examples. |
| `runs` | Specifies the command to run in the docker image. If `runs` is omitted, the<br>command specified in the `Dockerfile`'s `ENTRYPOINT` instruction will execute.<br>Use the `runs` attribute when the `Dockerfile` does not specify an `ENTRYPOINT`<br>or you want to override the `ENTRYPOINT` command. The `runs` attribute does not<br>invoke a shell by default. Using `runs: "echo $VAR"` will not print the value<br>stored in `$VAR`, but will instead print `\"\$VAR.\"`. To use environment<br>variables with the `runs` instruction, you must include a shell to expand<br>the variables, for example: `runs: ["sh", "-c", "echo $VAR"]`. If the value of `runs`<br>refers to a local script, the path is relative to the workspace folder (see [The workspace](#the-workspace) section below)|
| `args` | The arguments to pass to the command. This is an array of strings. For example,<br> `args: ["--flag", "--arg", "value"]`. If the value of `args`<br>refers to a local script, the path is relative to the workspace folder (see [The workspace](#the-workspace) section below). |
| `env` | The environment variables to set inside the container's runtime environment. If<br>you need to pass environment variables into a step, make sure it runs a command<br>shell to perform variable substitution. For example, if your `runs` attribute is<br>set to `["sh", "-c"]`, the value of `args` will be passed to `sh -c` and<br>executed in a command shell. Alternatively, if your `Dockerfile` uses an<br>`ENTRYPOINT` to run the same command (`"sh -c"`), `args` will execute in a<br>command shell as well. See [`ENTRYPOINT`](https://docs.docker.com/engine/reference/builder/#entrypoint) for more details. |
| `secrets` | Specifies the names of the secret variables to set in the runtime environment<br>which the container can access as an environment variable. For example,<br>`secrets: ["SECRET1", "SECRET2"]`. |
| `id` | Assigns an identifier to the step. By default, steps are asigned a numerid id<br>corresponding to the order of the step in the list, with `1` identifying<br>the first step. |
| `needs` | Identifies steps that must complete successfully before this step will be<br>invoked. It can be a string or an array of strings. |
| Attribute | Description |
| :--------- | :-------------------- |
| `uses` | **required** The Docker image that will be executed for that step. For example,<br>`uses: docker://node:10`. See **"Referencing images in a step"** section below for<br>more examples. |
| `id` | **optional** Assigns an identifier to the step. By default, steps are asigned a numerid id<br>corresponding to the order of the step in the list, with `1` identifying<br>the first step. |
| `runs` | **optional** Specifies the command to run in the docker image. If `runs` is omitted, the<br>command specified in the `Dockerfile`'s `ENTRYPOINT` instruction will execute.<br>Use the `runs` attribute when the `Dockerfile` does not specify an `ENTRYPOINT`<br>or you want to override the `ENTRYPOINT` command. The `runs` attribute does not<br>invoke a shell by default. Using `runs: "echo $VAR"` will not print the value<br>stored in `$VAR`, but will instead print `\"\$VAR.\"`. To use environment<br>variables with the `runs` instruction, you must include a shell to expand<br>the variables, for example: `runs: ["sh", "-c", "echo $VAR"]`. If the value of `runs`<br>refers to a local script, the path is relative to the workspace folder (see [The workspace](#the-workspace) section below)|
| `args` | **optional** The arguments to pass to the command. This is an array of strings. For example,<br> `args: ["--flag", "--arg", "value"]`. If the value of `args`<br>refers to a local script, the path is relative to the workspace folder (see [The workspace](#the-workspace) section below). Similarly to the `runs` attribute, if an envrionment<br>variable is being referenced, in order for this reference to be valid, a shell must be<br>be invoked (in the `runs` attribute) in order to expand the value of the variable. |
| `env` | **optional** The environment variables to set inside the container's runtime environment. If<br>you need to pass environment variables into a step, make sure it runs a command<br>shell to perform variable substitution. For example, if your `runs` attribute is<br>set to `["sh", "-c"]`, the value of `args` will be passed to `sh -c` and<br>executed in a command shell. Alternatively, if your `Dockerfile` uses an<br>`ENTRYPOINT` to run the same command (`"sh -c"`), `args` will execute in a<br>command shell as well. See [`ENTRYPOINT`](https://docs.docker.com/engine/reference/builder/#entrypoint) for more details. |
| `secrets` | **optional** Specifies the names of the secret variables to set in the runtime environment<br>which the container can access as an environment variable. For example,<br>`secrets: ["SECRET1", "SECRET2"]`. |
| `skip_pull` | **optional** Assume that the given container image already exist and skip pulling it. |

### Referencing images in a step

Expand Down Expand Up @@ -157,7 +147,6 @@ For example, let's look at a workflow that writes to a `myfile` in the
workspace:

```yaml
version: '1'
steps:
- uses: docker://alpine:3.9
args: [touch, ./myfile]
Expand Down Expand Up @@ -203,7 +192,6 @@ defines environment variables using the `env` attribute. For example,
you could set the variables `FIRST`, `MIDDLE`, and `LAST` using this:

```yaml
version: '1'
steps:
- uses: "docker://alpine:3.9"
args: ["sh", "-c", "echo my name is: $FIRST $MIDDLE $LAST"]
Expand Down Expand Up @@ -287,7 +275,6 @@ value for the `uses` attribute. This value instructs Popper to execute
the command or script given in the `runs` attribute. For example:

```yaml
version: '1'
steps:
- uses: "sh"
runs: ["ls", "-la"]
Expand All @@ -304,7 +291,6 @@ script specified in the `runs` attribute are NOT executed in a shell.
If you need a shell, you have to explicitly invoke one, for example:

```yaml
version: '1'
steps:
- uses: sh
runs: [bash, -c, 'sleep 10 && true && exit 0']
Expand Down Expand Up @@ -339,7 +325,6 @@ several nodes. You can get started with running Popper workflows through Slurm b

Let's consider a workflow `sample.yml` like the one shown below.
```yaml
version: '1'
steps:
- id: one
uses: docker://alpine:3.9
Expand Down

0 comments on commit aa62b01

Please sign in to comment.