smoke test latest cmx versions #4026
Conversation
cbodonnell
force-pushed
the
cbo/test-latest-cmx-versions
branch
5 times, most recently
from
6099129 to
0c27815
Compare
cbodonnell
force-pushed
the
cbo/test-latest-cmx-versions
branch
from
0c27815 to
b60a068
Compare
cbodonnell
force-pushed
the
cbo/test-latest-cmx-versions
branch
from
390266e to
9b917f7
Compare
| @@ -0,0 +1,802 @@ | |||
| name: compatibility-tests | |||
There was a problem hiding this comment.
Moved all of the "compatibility tests" over to this re-usable workflow so that we can potentially have a dual pipeline that runs Stable vs Alpha distros (AKS for example). This way we can test in a non-blocking way by making the alpha run of this workflow a non-required job.
There was a problem hiding this comment.
wouldn't it make more sense to have the CMX action mark a version as alpha or stable and then we could keep all these tests in build-test.yaml and then decide if a test should fail or not based on that?
There was a problem hiding this comment.
make sense - any ideas how we could make tests conditionally pass or fail if we went this route? i see continue-on-error, but that seems like it would still mark the job as failed?
There was a problem hiding this comment.
looking at the docs here, it seems it will allow the workflow to pass: https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#jobsjob_idcontinue-on-error
| }); | ||
|
|
||
| Object.keys(latestMinorVersions[distroName]).forEach((minorVersion) => { | ||
| let stage = 'stable'; |
There was a problem hiding this comment.
ideally, this should come from https://api.replicated.com/vendor/v3/cluster/versions
There was a problem hiding this comment.
that would be nice. maybe we can put that in as a feature request? i think there's maybe some value to the client determining their own definition of "stable", but agree that for this use-case it would make it easier. also trying to think ahead for if we make this a re-usable action. users should probably be able to pass an include or exclude to determine the versions they want or don't want, respectively. a conversation for later though...
What this PR does / why we need it:
This PR adds the remaining CMX distributions to our on-PR testing and also adds logic so that we will automatically inherit new CMX distros and versions that are added.
One larger change was to move all of the "compatibility tests" over to a separate re-usable workflow. This will make it easier for us to create two separate pipelines for stable vs alpha CMX distros. For example, we are excluding AKS for the moment because it is still alpha, but this could allow us to run those tests on AKS in a non-blocking way.
Which issue(s) this PR fixes:
Fixes #
Special notes for your reviewer:
Steps to reproduce
Does this PR introduce a user-facing change?
Does this PR require documentation?
NONE