Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Mention rolling update strategy is not appropriate for Crate major version upgrade #81

Open
c0c0n3 opened this issue Aug 7, 2020 · 5 comments
Labels
enhancement New feature or request

Comments

@c0c0n3
Copy link
Member

c0c0n3 commented Aug 7, 2020

Our Crate chart instantiates a K8s statefulset with a rolling update strategy. With this strategy in place, on detecting a new Crate image, K8s will delete and then recreate each pod with the new image, one at a time. So while upgrading, most of the time you'll be in a situation where some of the pods are running the old image whereas some others are on the new. Crate docs say this is fine only when upgrading to a new patch version whereas to upgrade to a new minor or major version you should first shut down the entire cluster, then upgrade images, and finally bring the new cluster live. This latter procedure isn't compatible with K8s rolling update strategy so we should warn users about it. For example we could mention it in the chart README.

@amotl
Copy link

amotl commented Nov 25, 2020

Dear @c0c0n3,

by following up on our conversation at orchestracities/ngsi-timeseries-api#384 (comment) ff. - FWIW, I would also like to reference crate/crate-operator#127 here, where I asked back about whether topics mentioned here might be implemented through the CrateDB Kubernetes Operator or not.

With kind regards,
Andreas.

@c0c0n3
Copy link
Member Author

c0c0n3 commented Nov 25, 2020

Dear @amotl, thank you sooo much for being so proactive. It really helps us alot. I added my comments to crate/crate-operator#127. Many thanks!!

@c0c0n3
Copy link
Member Author

c0c0n3 commented Nov 25, 2020

@chicco785 the bottom line for now is:

We should keep an eye on the above issue, since Crate Operator looks like a very nice option for our prod deployments.

@chicco785
Copy link
Contributor

@chicco785 the bottom line for now is:

  • K8s rolling upgrade isn't the proper way to upgrade to a new major version number, but should work fine between minor versions

yep, I think this was clear given current experience

that's very interesting, the issue for us is that we would need to be able to reuse existing pvc, or it will be a pain to migrate data. i suppose it works quite well also with fluxctl

We should keep an eye on the above issue, since Crate Operator looks like a very nice option for our prod deployments.

@c0c0n3
Copy link
Member Author

c0c0n3 commented Nov 25, 2020

would need to be able to reuse existing pvc

I think the Crate Operator will leave it be, if I understand the way it works but I could be wrong :-)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

3 participants