Decouple Helm Release with the Operator Release

[prudhvigodithi]

Description

Decouple Helm Release with the Operator Release.

• Update the release.yaml to exclude the helm release.
• Onboard new helm release workflow helm-release.yaml.
• The helm-release.yaml use the helm/chart-releaser-action which identifies the chart version and releases incrementally.
• The helm-release.yaml runs once the PR is merged to main and releases only once there is a change with chart version, inside the charts/ folder.

The expectation from the user is to create a PR with with updates changes, along with modifying the version of the chart, once merged the helm/chart-releaser-action will identify the new chart version and does a helm release.

Whenever there is an operator version release, the appVersion along with version has to be updated and thereafter the process is same helm/chart-releaser-action will identify the new chart version and does a helm release.

Issues Resolved

Coming from #830

Check List

• Commits are signed per the DCO using --signoff
• Unittest added for the new/changed functionality and all unit tests are successful
• Customer-visible features documented
• No linter warnings (make lint)

If CRDs are changed:

• CRD YAMLs updated (make manifests) and also copied into the helm chart
• Changes to CRDs documented

Please refer to the PR guidelines before submitting this pull request.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

[prudhvigodithi]

Adding @swoehrl-mw and @evheniyt to please take a look, once this is merged we can move forward with this PR #754 to refactor opensearch-cluster helm chart.

Once this PR is merged the idea is to rebase this PR #754, update the version and merge to main, then the helm/chart-releaser-action will take care of releasing the helm chart.

Thank you
@getsaurabh02 @peterzhuamazon

[peterzhuamazon]

So basically helm will always be latest HEAD?
While operator official release will be stable ones?

Thanks.

[prudhvigodithi]

So basically helm will always be latest HEAD? While operator official release will be stable ones?

Thanks.

Thanks Peter, the operator when released creates the docker image and other helm charts (part of the seperate releases) uses this operator version from appVersion.

It is the same mechanism that we use today for OpenSearch helm charts opensearch-project/helm-charts#569 where the appVersion is the OpenSearch version.

[swoehrl-mw]

I'm fine with doing this for the opensearch-cluster helm chart and can see the advantages of having a faster release cycle.
But for the opensearch-operator chart this is a problem with the current development process: If an operator feature requires changes to the chart (e.g. a new config option or changes to one of the CRDs) these are made in the same PR as the code changes. With this new release process we run a huge risk of releasing features in the chart that are not yet included in the operator code if someone makes a non-related change in the chart and wants to release that by increasing the version. This will only lead to problems.
So IMO the release and version of the operator chart should stay coupled with the operator release.

Also: You at least need to extend the CONTRIBUTING.md and docs/developing.md to explain the process, that users need to edit the changelog and how and when to increase helm chart versions.

[prudhvigodithi]

With this new release process we run a huge risk of releasing features in the chart that are not yet included in the operator code if someone makes a non-related change in the chart and wants to release that by increasing the version. This will only lead to problems.
So IMO the release and version of the operator chart should stay coupled with the operator release.

Thanks, @swoehrl-mw. I agree with your point, but I believe the maintainers are responsible for reviewing PRs and Helm chart changes. For example, in this PR #754, the change involved the opensearch-cluster, and the maintainers should be aware of code modifications made to the opensearch-operator Helm chart and decide if the chart version needs to be incremented.

This PR aims to standardize the Helm release process for both opensearch-cluster and opensearch-operator Helm charts, which would only require a version increment, and the workflow would handle the release. The timing and coordination of version increments should be managed by the PR authors and maintainers to ensure smooth chart releases.

In some cases, like a critical/small fix, we may need to release the opensearch-operator Helm chart independently, without waiting for the opensearch-operator release and this process (by this PR) should be helpful and unblock the chart release.

Thank you
Adding @getsaurabh02 @salyh @pchmielnik @jochenkressin @dblock @peterzhuamazon

[swoehrl-mw]

@prudhvigodithi
I still think this will lead to problems, to be honest I don't think we have the best track record there with keeping stability and thoroughly checking PRs.

IMO it is the lesser evil to have different release processes than making it easy to break things.

In some cases, like a critical/small fix, we may need to release the opensearch-operator Helm chart independently, without waiting for the opensearch-operator release and this process (by this PR) should be helpful and unblock the chart release.

Should this actually happen of doing a release of the opeator+chart without code changes, that is not really an argument for me.

[prudhvigodithi]

@swoehrl-mw any other suggestions on how to move forward with this PR #754 without decoupling the releases?

that is not really an argument for me.

Again we cant release an operator for every small change in the helm chart (and helm chart should not wait until the operator is released), IMO its totally worth to decouple both the 2 products, the operator and the helm chart.

[prudhvigodithi]

Following, @swoehrl-mw can you please suggest the next steps here? I want to Decouple Helm Release with the Operator Release because of #872 (comment) and unblock the PR #754.
Thank you
@getsaurabh02 @peterzhuamazon

[swoehrl-mw]

@prudhvigodithi I would suggest to have the opensearch-cluster helm chart with the release process as you suggest it in this PR and have operator docker + opensearch-operatorhelm chart with the release process as it is currently. That way #754 would be unblocked. Releases for the operator chart would still be slow, but as I said in an earlier comment that chart is tightly coupled with operator features so they should really be released only together.

[evheniyt]

@prudhvigodithi for me this sounds reasonable, could we start moving forward with decoupling for opensearch-cluster?

[prudhvigodithi]

Sure @evheniyt let me finalize the PR with just decoupling opensearch-cluster helm chart.

[prudhvigodithi]

Coming from https://github.com/helm/chart-releaser?tab=readme-ov-file#dealing-with-charts-that-have-dependencies leveraged charts_dir to release opensearch-operator helm chart with the common operator release and opensearch-cluster on merge to main with version increment.