Example with an private maven repository

[benkeil]

Does someone has an example how to include a library with inside a private maven repository?
I want to add common tasks/actions I need for my workflows in a library.

[krzema12]

Hi Ben! I'm not quite sure I understand what you're trying to achieve. Could you elaborate, perhaps include some pseudocode of what you'd like to do? In general, the main way of using this library doesn't deal with build system, one can even use it outside of the JVM world. As helper questions: is what you want to do possible with Gradle or public repositories?

[benkeil]

I want to template everything. I have different kinds of workflows (library, Microservices, lambda, infrastructure, ...). In my case, all workflows use the same programming language (kotlin) and therefore have some steps in common. I want to define now in a private library (private github maven package) steps. E.g. KotlinBuildStep, KotlinTestStep, TerraformDeploy, GithubCheckout, etc.
I want to reference these predefined steps then later in a concrete repository.

a default lambda deployment:

@file:DependsOn("it.krzeminski:github-actions-kotlin-dsl:0.29.0")
@file:Repository("https://maven.pkg.github.com/my-company/github-actions-library") // needs credentials
@file:DependsOn("my.company:github-actions-library:1.0.0")

import my.company.github-actions-library.workflows.KotlinLambdaDeployWorkflow
...

KotlinLambdaDeployWorkflow(name = "foobar-lambda").writeToFile()

a custom deployment:

@file:DependsOn("it.krzeminski:github-actions-kotlin-dsl:0.29.0")
@file:Repository("https://maven.pkg.github.com/my-company/github-actions-library") // needs credentials
@file:DependsOn("my.company:github-actions-library:1.0.0")

import my.company.github-actions-library.jobs.KotlinTestJob
import my.company.github-actions-library.steps.KotlinBuildStep
...

workflow(
    name = "build",
    on = listOf(
        Push(branches = listOf("main")),
        PullRequest(),
    ),
    sourceFile = __FILE__.toPath(),
) {
    KotlinTestJob()

    // basically how KotlinTestJob() could look like
    job(id = "build", runsOn = UbuntuLatest) {
        KotlinBuildStep()
        KotlinTestStep()
    }

    job(id = "deploy-dev", runsOn = UbuntuLatest) {
        TerraformDeploy(stage = "dev")
    }
    
    job(id = "deploy-live", runsOn = UbuntuLatest) {
        TerraformDeploy(stage = "live")
    }
}.writeToFile()

[krzema12]

Ok, I think I understand what you want to do, but I don't understand what the problematic part is. Is it about authentication with your private Maven repo from a Kotlin script, as your inline comment says?

[benkeil]

Yes, I didn't used kotlin script before and I'm unable to authenticate to the repo. Or in general, how would I extend your DSL?

[benkeil]

How would I extend your library?

I tried the following:

import it.krzeminski.githubactions.actions.actions.CheckoutV3
import it.krzeminski.githubactions.domain.ExternalActionStep
import it.krzeminski.githubactions.domain.Job
import it.krzeminski.githubactions.domain.JobOutputs
import it.krzeminski.githubactions.domain.RunnerType
import it.krzeminski.githubactions.domain.triggers.Push
import it.krzeminski.githubactions.dsl.workflow
import it.krzeminski.githubactions.yaml.toYaml
import java.nio.file.Path
import java.nio.file.Paths

fun kotlinLambdaBuildWorkflow(sourceFile: Path) = workflow(
  name = "Test workflow",
  on = listOf(Push()),
  sourceFile = sourceFile,
) {
  job(id = "test_job", runsOn = RunnerType.UbuntuLatest) {
    uses(name = "Check out", action = CheckoutV3())
    checkoutStep()
    run(name = "Print greeting", command = "echo 'Hello world!'")
  }
  kotlinCheckJob()
}

fun checkoutStep() = ExternalActionStep(
  id = "checkout",
  action = CheckoutV3(fetchDepth = CheckoutV3.FetchDepth.Value(0)),
)

fun kotlinCheckJob() = Job(
  id = "check",
  runsOn = RunnerType.UbuntuLatest,
  steps = listOf(),
  outputs = JobOutputs.EMPTY,
)

fun main(args: Array<String>) {
  val sourceFile = Paths.get("").toAbsolutePath()
  val workflow = kotlinLambdaBuildWorkflow(sourceFile = sourceFile)
  println(workflow.toYaml(addConsistencyCheck = false))
}

[krzema12]

We have two problems here:

1. Using a private Maven repository with credentials from a Kotlin script. I don't know how to do it, and it's not a problem of this DSL directly, but a more generic problem related to Kotlin scripting. It would be good to ask on Kotlin's Slack, and if this feature is missing, request it via YouTrack. Please share any outcomes here with us.
2. Extending the DSL. The DSL makes heavy use of so called type-safe builders where lambdas with receivers are used. To extract a certain part of GitHub's workflow or job, you can create an extension function with the corresponding type, see e. g. https://github.com/krzema12/github-workflows-kt/blob/b7bb01e88c5fe68feec2e0f3788310aec8111e8b/.github/workflows/_shared.main.kts#L9 and a related discussion: Extracting Common Code #394

[Vampire]

According to https://youtrack.jetbrains.com/issue/KT-27701/Add-credentials-support-to-maven-and-ivy-resolvers-used-in-kotlin-scripts it should be like

@file:DependsOn("group:artifact:version", options = ["username=<user>", "password=<password>"])

if you use 1.6.0 or newer.

[benkeil]

I have now this

import it.krzeminski.githubactions.actions.actions.CheckoutV3
import it.krzeminski.githubactions.domain.JobOutputs
import it.krzeminski.githubactions.domain.RunnerType
import it.krzeminski.githubactions.domain.triggers.Push
import it.krzeminski.githubactions.dsl.JobBuilder
import it.krzeminski.githubactions.dsl.WorkflowBuilder
import it.krzeminski.githubactions.dsl.workflow
import it.krzeminski.githubactions.yaml.toYaml
import java.nio.file.Path
import java.nio.file.Paths

fun kotlinLambdaBuildWorkflow(sourceFile: Path) = workflow(
  name = "Test workflow",
  on = listOf(Push()),
  sourceFile = sourceFile,
) {
  kotlinCheckJob("check-1", "world")
  kotlinCheckJob("check-2", "mars")
}

fun WorkflowBuilder.kotlinCheckJob(id: String = "check", text: String) {
  job(id = id, runsOn = RunnerType.UbuntuLatest) {
    checkoutStep()
    helloStep(text)
  }
}

fun JobBuilder<*>.checkoutStep() {
  uses(action = CheckoutV3(fetchDepth = CheckoutV3.FetchDepth.Value(0)))
}

fun JobBuilder<*>.helloStep(text: String) {
  run(name = "Print greeting", command = "echo 'Hello $text!'")
}

fun main(args: Array<String>) {
  val sourceFile = Paths.get("").toAbsolutePath()
  val workflow = kotlinLambdaBuildWorkflow(sourceFile = sourceFile)
  println(workflow.toYaml(addConsistencyCheck = false))
}

I have the feeling that it will get messy if I have a lot of extensions. Imagine 15 steps per technology and and platform (kotlin-lambda, rust-lambd, kotlin-ec2, rust-ec2, ...). Can I compose everything without the DSL?

[krzema12]

I don't understand what you mean by composing without the DSL. Is there a problem in using it, or rather scalable structure of your code as new steps are added? It's hard for me to recommend anything concrete since we don't have the code with 15 steps, but since we're using a general-purpose language, you can try to remove duplication to the max. IMO having this ability is still good comparing to YAML, not quite sure what more you'd expect.

[krzema12]

@benkeil do you still need some help with using the private Maven repo? Could you share for future reference what code worked for you? If some answer did the job, please "Mark as answer".