Extracting Common Code

[scottbisaillon]

I finally got around to creating my first workflow and the developer experience was definitely better than from writing it in just yaml, so now I am looking to convert more of my workflows over.

A lot of them have common steps, or blocks of steps, so I am trying to figure out the best way to extract the common parts and use them in multiple workflows.

For example, I have this CommandStep:

run(name = "Install Tools", command = expr { github.workspace } + "/.github/workflows/scripts/install-tools-flutter.sh")

This may be due to my lack of experience in kotlin, but my first instinct was to make it a function:

fun installTools(): CommandStep {
    return CommandStep(
        id = "install-tools",
        name = "Install Tools",
        command = expr { github.workspace } + "/.github/workflows/scripts/install-tools-flutter.sh"
    )
}

and then in the workflow, use it like:

job(...) {
  // ...other steps
  installTools()
  // ...other steps
}

After looking into the library code a little, I realize that the functions used in the builder actually append the step to the job's current steps, so what I tried doesn't work, as when I run the script, the yaml outputted doesn't have that step in the job.

Looking for some advice on how to go about this scenario as well as potentially extracting a series of common steps, but the latter is not as important if its actually more cumbersome to do so.

Thanks for any help and I appreciate the work on this library!

[krzema12]

Hi Scott, thanks for the kind words - I'm happy that you see the value the library gives :)

Looking at your case, let's go step by step. Let's say you have such workflow script:

#!/usr/bin/env kotlin
@file:DependsOn("it.krzeminski:github-actions-kotlin-dsl:0.24.0")

import it.krzeminski.githubactions.actions.actions.CheckoutV3
import it.krzeminski.githubactions.domain.RunnerType.UbuntuLatest
import it.krzeminski.githubactions.domain.triggers.PullRequest
import it.krzeminski.githubactions.domain.triggers.Push
import it.krzeminski.githubactions.dsl.expressions.Contexts.github
import it.krzeminski.githubactions.dsl.expressions.expr
import it.krzeminski.githubactions.dsl.workflow
import it.krzeminski.githubactions.yaml.writeToFile

workflow(
    name = "Example workflow",
    on = listOf(
        Push(branches = listOf("main")),
        PullRequest()
    ),
    sourceFile = __FILE__.toPath()
) {
    job(
        id = "test-job",
        runsOn = UbuntuLatest
    ) {
        uses(CheckoutV3())
        run(
            name = "Install Tools",
            command = expr { github.workspace } + "/.github/workflows/scripts/install-tools-flutter.sh")
    }
}.writeToFile()

If you hold Ctrl/Comand and right-click on run function in IntelliJ, you'll notice that it not only creates an object of class CommandStep, it also changes the state of the job builder (job = job.copy(steps = job.steps + newStep)). That's why your approach doesn't produce the command step in the YAML.

Fortunately, IntelliJ is one of the best IDEs out there, and its refactoring utilities are pretty awesome. Here, to solve the problem, it's enough to select the whole invocation of run, right-click on it, and select Refactor > Function.... Then we get this:

#!/usr/bin/env kotlin
@file:DependsOn("it.krzeminski:github-actions-kotlin-dsl:0.24.0")

import it.krzeminski.githubactions.actions.actions.CheckoutV3
import it.krzeminski.githubactions.domain.RunnerType.UbuntuLatest
import it.krzeminski.githubactions.domain.triggers.PullRequest
import it.krzeminski.githubactions.domain.triggers.Push
import it.krzeminski.githubactions.dsl.JobBuilder
import it.krzeminski.githubactions.dsl.expressions.Contexts.github
import it.krzeminski.githubactions.dsl.expressions.expr
import it.krzeminski.githubactions.dsl.workflow
import it.krzeminski.githubactions.yaml.writeToFile

workflow(
    name = "Example workflow",
    on = listOf(
        Push(branches = listOf("main")),
        PullRequest()
    ),
    sourceFile = __FILE__.toPath()
) {
    job(
        id = "test-job",
        runsOn = UbuntuLatest
    ) {
        uses(CheckoutV3())
        extracted()
    }
}.writeToFile()

fun JobBuilder.extracted() {
    run(
        name = "Install Tools",
        command = expr { github.workspace } + "/.github/workflows/scripts/install-tools-flutter.sh")
}

What the IDE created is an extension function, which apparently is also present in Dart. Now, the run function operates on the this context present in the extracted method, which in this case is the JobBuilder.

This works equally well for extracting a series of steps. It's like cutting out a part of the workflow and putting it in the extension functions. There will be simply multiple calls to run or uses, or whatever you need.

---

As a bonus, let me give you a hint how to extract such functions to a separate file. The file would be named e.g. common.main.kts (the .main.kts extension is important) and would contain:

@file:DependsOn("it.krzeminski:github-actions-kotlin-dsl:0.24.0")

import it.krzeminski.githubactions.dsl.JobBuilder
import it.krzeminski.githubactions.dsl.expressions.expr

fun JobBuilder.extracted() {
    run(
        name = "Install Tools",
        command = expr { github.workspace } + "/.github/workflows/scripts/install-tools-flutter.sh")
}

Then it's enough to reference it from the workflow script by writing:

@file:Import("common.main.kts")

It works fine when you run the script, but unfortunately, IDE support is not yet complete. What I see in the workflow script is the call to extracted() marked with red, as if it couldn't find it. I think I saw a workaround for it, but I cannot recall it now. Hopefully Kotlin Script will get more attention and such imperfections will be fixed.

If you have any further questions, just ask :)

[scottbisaillon]

What the IDE created is an extension function, which apparently is also present in Dart. Now, the run function operates on the this context present in the extracted method, which in this case is the JobBuilder.

Ah yes! I am familiar with extension methods and they are something I use quite often. I actually did try this out, but my issue was forgetting about using the already defined run and uses methods in the builder. That makes more sense now.

As a bonus, let me give you a hint how to extract such functions to a separate file. The file would be named e.g. common.main.kts (the .main.kts extension is important) and would contain:

Thanks for this, I was exploring putting the common code into a separate file but ran into the exact problem that you stated. It is unfortunate and I do hope better IDE support comes in the future. I stumbled upon the JetBrains issue for this at one point, but couldn't find it again to see if there was any other traction on it.

I am contemplating just putting all my workflows in a single file for now and then writing them out to the appropriate files, at least until there is better support for imports.

Thanks for the detailed response, its greatly appreciated!