Upgrade cronjobs from batch/v1beta to batch/v1

[anuragpathak21]

Version 1.21 of kubernetes promoted cron jobs from batch/v1beta to batch/v1

Upgrade cronjobs from batch/v1beta to use batch/v1

[onobc]

Some background info on the current API (v125)

• https://kubernetes.io/docs/reference/using-api/deprecation-guide/#cronjob-v125

[anuragpathak21]

@onobc Due to removal of batch/v1beta1 from V1.25 we are unable to use SCDF in our cluster with version 1.25.

[onobc]

@anuragpathak21 - I understand. Sorry, if my comment w/ link (but w/o any context) was confusing. It was meant as an internal note to lookup the latest details of the API. I meant to add some context w/ the link.

We are looking into this issue currently and should have a direction in the next couple days.

[markpollack]

I am waiting to hear back from our commercial customers and we will update you shortly. I'd expect we make the update in the next point release FWIW.

Just out of curiosity, are you aware of the features that are available only in our commercial offering?

https://docs.vmware.com/en/VMware-Spring-Cloud-Data-Flow-for-Kubernetes/1.5/scdf-k8s/GUID-pro-features-index.html

[onobc]

We are unable to make this change in 2.10 as we can not exclude k8s version 1.19,1.20 in SCDF 2.10.

We will make this change in 2.11 and the k8s compatibility will look as follows:

• 2.11 will support k8s 1.21 - 1.25
• 2.10 supports k8s < 1.25

The next question is when will 2.11 be available. We are working on it currently and will firm up some milestone dates soon.

[dclangan]

@onobc What is the best way to get updates regarding timing for 2.11 release?

[onobc]

Hi @dclangan .

This week the team will assign issues and a date to the 2.11.0 milestone.

The best way to "track" is the date on the milestone and the issues still in open status on the milestone.

Somewhat unrelated - but good to know... note that we have:

• created a mini-mono-repo for all deployers Upgrade cronjobs from batch/v1beta to batch/v1 #529
• converted https://github.com/spring-cloud/spring-cloud-dataflow into a mini-mono-repo for all other SCDF pieces.

[ganto]

Any update on the date for the 2.11.0 release? We are currently stuck rolling out OpenShift updates due to this issue.

[onobc]

Hi @ganto
We got pulled off on other issues last week and did not get a chance to firm up the dates. We should have a date set in the next few days.

Thanks

[onobc]

We have an estimated release date of 5/17/2023.

[v1bh0r]

Hello, @onobc, is there a revised ETA on this release?

[onobc]

Hello @v1bh0r , yes the date was moved to 2023-06-20. I mentioned in my reply on the duplicate issue but it is a good thing to callout here as the ETA was previously mentioned above. Thanks.

The fix is in the 2.11.0-SNAPSHOT if you want to try it out.

[v1bh0r]

Hello @onobc , we are eagerly awaiting the 2.11 release today. Just in case the release gets delayed, can you point me in the right direction to build the docker image for the 2.11 codebase and deploy it as a part of the cluster deployed using the bitnami charts?

[dhpizza]

Hi @onobc
this is still not released, correct?
What is the new date to have this available? We are using the bitnami chart, so would need a released 2.11.
Will this spring-cloud/spring-cloud-deployer#378 be in this milestone ?

Thank you 🙏

[cppwfs]

Sorry for the delay in response.
It will be present in the milestone for 2.11.

[dhpizza]

Hello @onobc , we are eagerly awaiting the 2.11 release today. Just in case the release gets delayed, can you point me in the right direction to build the docker image for the 2.11 codebase and deploy it as a part of the cluster deployed using the bitnami charts?

Hi @v1bh0r

Did you get that working, i.e. using the 2.11-Snapshot with the bitnami chart?

[v1bh0r]

@dhpizza: We have chosen an unconventional workaround - We are scheduling the jobs are regular k8s cron jobs outside of the SCDF.

[dclangan]

@onobc Noticed that 2.11.0 no longer has a due date - with the looming EOL for GKE 1.24 being 2 months away, is there a revised ETA for 2.11.0, or perhaps a known stable pre-release snapshot that may be recommended for use now?

Thanks!

[onobc]

Hi @dclangan ,
We removed it as it was not doing much good saying that it was 2 weeks past due. While that is 100% true, it is not helpful for those waiting for it and wanting to know the ETA. We will replace it w/ an accurate ETA today in our team meeting.

Thanks for your patience.

[dbasurco]

Good morning @onobc ,
Do you have a planned release date? we are coming out of EKS version 1.23 and moving to 1.24 with an EOL date so close doesn't fit very well. Thank you very much!

[anuragpathak21]

7 months and the fix is still not in. This raise doubts about future support of such issues. Should we start looking at alternative to SCDF?

[onobc]

@dbasurco @anuragpathak21 thanks for your continued patience. The date is hardened and is 2023-09-14. The Dataflow 2.11.0 and Deployer 2.9.0 milestone are updated accordingly.

I understand your frustration @anuragpathak21 and I will explain a bit about the delay. Currently the Deployer and Dataflow releases are dependent. The k8s side was ready to go quite a while ago. The Dataflow side however, had large changes to support Spring Boot 2 and 3 | Spring Cloud Task 2 and 3 | Spring Cloud Batch 4 and 5 workloads on the same server. These changes took longer than expected as we ran into a few snafus along the way. It is now working and we feel the above date is realistic and will be accomplished. Going forward we will either break the Deployer and Dataflow releases up or we will be careful about what we put into a release when there are high priority competing interests in the two pieces.

If this were a critical bug or a critical security issue we would have made sure it was addressed - regardless of the situations described above. This one was (arguably) somewhat of a bug and somewhat of a feature request. It should have not taken this long and we will make sure to plan better going forward.

[dbasurco]

thank you for the explanation! we will wait until the 14th.

[anuragpathak21]

Thanks @dbasurco for the detail explanation. We will wait for the release on 14th.

[onobc]

@dbasurco @anuragpathak21 we were on track to release today and then we found a bug during staging verification. We had a team meeting to decide if we wanted to fix this before the release - and the vote was yes.

Good news is it should be a straight forward fix and we should be able to release tomorrow 09/15 or at the latest Monday 09/18.

Backing issue: spring-cloud/spring-cloud-dataflow#5467

[dbasurco]

Hello @onobc i understand there is some new issues :( . Do you know more information about the new release date? thanks for the effort! Best regards,

[rallepalle]

Hi @onobc we are waiting for the new release 2.11.0 since your announcement in may. Now it should appear on 09/18. Unfortunately, it is currently a critical point for us because in a few days a k8s upgrade is pending and then release 2.10.x is no longer supported! We need therefore a safe statement when release 2.11.0 is published! thanks anyway for your great work!

[onobc]

The release is ready and we are dealing w/ CI infrastructure issues and it should be out today. We have been working around the clock since last Friday to get this to Maven Central. It should be today.

Everyone, the release will be out shortly. This will be the last of the "where is the release" messages I answer on this closed ticket.

Thanks

[onobc]

I wanted to follow up here first and thank everyone for their patience. The bits are on Maven Central and the docs are updated. We will follow up w/ the blog and release notes shortly.

Ref docs

• https://docs.spring.io/spring-cloud-dataflow/docs/2.11.0/reference/htmlsingle/#appendix-boot3

Libs

• https://repo1.maven.org/maven2/org/springframework/cloud/spring-cloud-dataflow-server/2.11.0/
• https://repo1.maven.org/maven2/org/springframework/cloud/spring-cloud-skipper-server/2.11.0/
• etc...

Containers

• https://hub.docker.com/r/springcloud/spring-cloud-dataflow-server/tags
• https://hub.docker.com/r/springcloud/spring-cloud-skipper-server/tags
• etc..