Add a multi-cloud test profile

[abhi18av]

Overview

Hi team 👋

This draft PR addresses the changes identified in Slack conversation regarding Azure tests for all pipelines.

If someone could please offer an early feedback, I can finalize this PR and then open for merge 🙏

PR checklist

• This comment contains a description of changes (with reason)
• CHANGELOG.md is updated
• If you've fixed a bug or added code that should be tested, add tests!
• Documentation in docs is updated

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Comparison is base (3a3ecbc) 74.95% compared to head (aa28e16) 74.89%.
Report is 28 commits behind head on dev.

Additional details and impacted files

@@            Coverage Diff             @@
##              dev    #2544      +/-   ##
==========================================
- Coverage   74.95%   74.89%   -0.07%     
==========================================
  Files          85       85              
  Lines        9231     9220      -11     
==========================================
- Hits         6919     6905      -14     
- Misses       2312     2315       +3     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[mirpedrol]

I think it's looking good!
I only commented on some details.
Also, don't forget to update the changelog :)

[mirpedrol]

We have to add {%- raw %} and {% endraw %} at the beginning and end of double curly brackets to avoid errors when generating the pipeline from a template. You can see examples on other GHA such as awstest.yml

[abhi18av]

Quick summary for @mirpedrol for a follow-up

1. I've minimized the surface area to only cover aws block right now 0d3aeb2

2. The test seems to be passing but there is a warning regarding jinja (still 😞 )

[mirpedrol]

I think the if statement on line 18 is the one with curly brackets left, so if you move the comment from line 20 up this should be fixed 🙂

And I noticed while testing this that the curly brackets in commented out lines need to be skipped too 😮 so the same change should be made for the other jobs.

[adamrtalbot]

Instead of running multiple jobs, we could use a matrix include block like so?

      matrix:
        include:
          - profile: test
            platform: aws
            workdir: s3://${{ secrets.AWS_S3_BUCKET }}/work/sarek/work-${{ github.sha }}/${{ matrix.profile }}
            outdir: "s3://${{ secrets.AWS_S3_BUCKET }}/sarek/results-${{ github.sha }}/test/"
          - profile: test
            platform: azure
            workdir: ${{ TOWER_BUCKET_AZURE }}/work/sarek/work-${{ github.sha }}/${{ matrix.profile }}
            outdir: "${{ TOWER_BUCKET_AZURE }}/sarek/results-${{ github.sha }}/test/"

with something like this (untested):

      - name: ${{ matrix.platform }} Launch
        uses: seqeralabs/action-tower-launch@v2
        with:
          run_name: sarek_${{ matrix.profile }}_${{ matrix.platform }}
          workspace_id: ${{ secrets.TOWER_WORKSPACE_ID }}
          access_token: ${{ secrets.TOWER_ACCESS_TOKEN }}
          compute_env: ${{ secrets.TOWER_COMPUTE_ENV }}
          revision: ${{ github.sha }}
          workdir: s3://${{ secrets.AWS_S3_BUCKET }}/work/sarek/work-${{ github.sha }}/${{ matrix.profile }}
          parameters: |
            {
              "hook_url": "${{ secrets.MEGATESTS_ALERTS_SLACK_HOOK_URL }}",
              "outdir": "${{ matrix.outdir }}"
            }
          profiles: ${{ matrix.profile }}

[adamrtalbot]

Instead of running multiple jobs, we could use a matrix include block like so?

      matrix:
        include:
          - profile: test
            platform: aws
            workdir: s3://${{ secrets.AWS_S3_BUCKET }}/work/sarek/work-${{ github.sha }}/${{ matrix.profile }}
            outdir: "s3://${{ secrets.AWS_S3_BUCKET }}/sarek/results-${{ github.sha }}/test/"
          - profile: test
            platform: azure
            workdir: ${{ TOWER_BUCKET_AZURE }}/work/sarek/work-${{ github.sha }}/${{ matrix.profile }}
            outdir: "${{ TOWER_BUCKET_AZURE }}/sarek/results-${{ github.sha }}/test/"

with something like this (untested):

      - name: ${{ matrix.platform }} Launch
        uses: seqeralabs/action-tower-launch@v2
        with:
          run_name: sarek_${{ matrix.profile }}_${{ matrix.platform }}
          workspace_id: ${{ secrets.TOWER_WORKSPACE_ID }}
          access_token: ${{ secrets.TOWER_ACCESS_TOKEN }}
          compute_env: ${{ secrets.TOWER_COMPUTE_ENV }}
          revision: ${{ github.sha }}
          workdir: s3://${{ secrets.AWS_S3_BUCKET }}/work/sarek/work-${{ github.sha }}/${{ matrix.profile }}
          parameters: |
            {
              "hook_url": "${{ secrets.MEGATESTS_ALERTS_SLACK_HOOK_URL }}",
              "outdir": "${{ matrix.outdir }}"
            }
          profiles: ${{ matrix.profile }}

Problem with this is you can't use secrets in the matrix. Dang it.

[abhi18av]

Putting this on halt till there's a consensus on https://nfcore.slack.com/archives/C04514S9BV5/p1701253055943449