From 4ab7ecaf7b22d03c631cf2944ea3bbe4573fac44 Mon Sep 17 00:00:00 2001 From: Kevin W Monroe Date: Thu, 29 Feb 2024 14:00:41 -0600 Subject: [PATCH] 1.29 upgrade notes (#834) * refresh generic upgrade page * blurb about ceph and gaps in the main release notes * refresh ceph page with new ceph-csi usage * refresh specific 1.29 upgrade page * tweak o7k integration page * blurb about o7k and cloud integrators in the main release notes * Update pages/k8s/1.29/release-notes.md Co-authored-by: Louise K. Schmidtgen <107889048+louiseschmidtgen@users.noreply.github.com> * full stops look funny in lists * sync general and specific release notes * add big warning at the top of the upgrade page * Update pages/k8s/1.29/upgrading.md --------- Co-authored-by: Louise K. Schmidtgen <107889048+louiseschmidtgen@users.noreply.github.com> Co-authored-by: Nick Veitch --- pages/k8s/1.29/release-notes.md | 47 +++++++++- pages/k8s/1.29/upgrading.md | 46 +++++---- pages/k8s/ceph.md | 144 +++++++++++++---------------- pages/k8s/openstack-integration.md | 44 +++++---- pages/k8s/release-notes.md | 50 +++++++++- pages/k8s/upgrading.md | 30 +++--- 6 files changed, 216 insertions(+), 145 deletions(-) diff --git a/pages/k8s/1.29/release-notes.md b/pages/k8s/1.29/release-notes.md index 57145bca..b4b4e109 100644 --- a/pages/k8s/1.29/release-notes.md +++ b/pages/k8s/1.29/release-notes.md @@ -15,7 +15,7 @@ toc: False --- # 1.29 -### February 23, 2024 - `charmed-kubernetes --channel 1.29/stable` +### February 12, 2024 - `charmed-kubernetes --channel 1.29/stable` The release bundle can also be [downloaded here](https://raw.githubusercontent.com/charmed-kubernetes/bundle/main/releases/1.29/bundle.yaml). @@ -24,8 +24,8 @@ The release bundle can also be [downloaded here](https://raw.githubusercontent.c ### Charmed Operator Framework (Ops) We're pleased to announce the completion of the Charmed Kubernetes refactor that began -last year. Charms have moved from the `reactive` and `pod-spec` styles to the `ops` -framework in order to enable access to common charm libraries, gain better Juju support, +last year. Core charms have moved from the `reactive` and `pod-spec` styles to the `ops` +framework. This shift aims to enable access to common charm libraries, gain better Juju support, and provide a more consistent charming experience for community engagement. ### Out of the box monitoring enhancements @@ -38,6 +38,20 @@ observability tools. This release expands our COS integration so that it includes rich monitoring for the control plane and worker node components of Charmed Kubernetes. +### Ceph CSI + +Ceph CSI resource management has been decoupled from the `kubernetes-control-plane` +charm. All new deployments should leverage the [ceph-csi][] charm for Ceph storage +provisioning, including support for CephFS. See the [updated documentation][ceph] for +details on deploying Charmed Kubernetes with Ceph support. + +### OpenStack integration + +OpenStack capabilities (including cinder storage and cloud provider) have been decoupled +from the `kubernetes-control-plane` charm. All new deployments should leverage the new +`openstack-integrator`, `openstack-controller-manager`, and `cinder-csi` charms. See the +[updated documentation][openstack] for more details. + ### NVIDIA GPU Operator The new [nvidia-gpu-operator][] charm simplifies the management of NVIDIA GPU resources @@ -73,7 +87,29 @@ support and is offered as an alternative to the default Calico CNI. All bug fixes and other feature updates in this release can be found at [the launchpad milestone page for 1.29](https://launchpad.net/charmed-kubernetes/+milestone/1.29). -## Known Issues + + +## Notes and Known Issues + +### Integration gaps + +While Charmed Kubernetes core charms have been rewritten in the `ops` framework, some +integrations with other charms in the Juju ecosystem are not available in this release. +If your deployment depends on the following, we recommend remaining on Charmed Kubernetes +1.28 until compatible support is provided in a subsequent release: + +- AWS IAM authentication: [aws-iam](https://charmhub.io/aws-iam) +- Keystone authentication: [keystone](https://charmhub.io/keystone) +- Vault storage: [vault](https://charmhub.io/vault) + +### Cloud integrators + +Direct integration between `[aws|azure|gcp|openstack]-integrator` charms and +`kubernetes-control-plane` has been removed in this release. The recommended method for +enabling native cloud features is to use the respective out-of-tree cloud provider +charms. See the cloud-specific documentation for details. + +### Bugs A list of known bugs scheduled to be fixed in the first maintenance release can be found on the [1.29+ck1 milestone page](https://launchpad.net/charmed-kubernetes/+milestone/1.29+ck1). @@ -93,6 +129,9 @@ relevant sections of the [upstream release notes][upstream-changelog-1.29]. [rel]: /kubernetes/docs/release-notes +[ceph-csi]: https://charmhub.io/ceph-csi?channel=1.29/stable +[ceph]: /kubernetes/docs/ceph +[openstack]: /kubernetes/openstack-integration [nvidia-gpu-operator]: https://charmhub.io/nvidia-gpu-operator?channel=1.29/stable [gpu-workers]: /kubernetes/docs/gpu-workers [install-local]: /kubernetes/docs/install-local diff --git a/pages/k8s/1.29/upgrading.md b/pages/k8s/1.29/upgrading.md index aecee935..57fb6115 100644 --- a/pages/k8s/1.29/upgrading.md +++ b/pages/k8s/1.29/upgrading.md @@ -13,9 +13,18 @@ layout: [base, ubuntu-com] toc: False --- +
+
+ Caution: +

This release includes topology changes and new best practices for integrating Charmed Kubernetes with other Juju ecosystem solutions. Be sure to read and understand the *What's new* section of the 1.29 release notes prior to upgrading your cluster.
+
+ Additionally, some features from previous Charmed Kubernetes releases are not yet available in this release. If you rely on a component identified as an *Integration gap* in the Notes and Known Issues section of the release notes, remain on release 1.28 (or earlier) and do not upgrade to 1.29 at this time.

+
+
+ It is recommended that you keep your **Kubernetes** deployment updated to the latest available stable version. You should also update the other applications which make up **Charmed Kubernetes**. Keeping up to date ensures you have the latest bug-fixes and security patches for smooth operation of your cluster. -You can check the latest release version on the [Kubernetes release page on GitHub][k8s-release]. +New minor versions of **Kubernetes** are set to release three times per year. You can check the latest release version on the [Kubernetes release page on GitHub][k8s-release].
@@ -52,7 +61,7 @@ You should also make sure: - Your Juju client and controller/models are running the same, stable version of Juju (see the [Juju docs][juju-controller-upgrade]). - You read the [Upgrade notes][notes] to see if any caveats apply to the versions you are upgrading to/from. - You read the [Release notes][release-notes] for the version you are upgrading to, which will alert you to any important changes to the operation of your cluster. -- You read the [Upstream release notes](https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG/CHANGELOG-1.29.md#deprecation) for details of deprecation notices and API changes for Kubernetes 1.29 which may impact your workloads. +- You read the [Upstream release notes](https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG/CHANGELOG-1.29.md#deprecation) for details of Kubernetes deprecation notices and API changes that may impact your workloads. It is also important to understand that **Charmed Kubernetes** will only upgrade, and migrate if necessary, components relating specifically to elements of @@ -68,7 +77,7 @@ deprecated APIs.
-## Upgrading the Machine's Series (required for machines currently running 18.04(Bionic)) +## Upgrading the series (required for machines currently running Ubuntu 18.04) All of the charms support [upgrading machine series via Juju](https://juju.is/docs/juju/manage-machines#heading--upgrade-a-machine). As each machine is upgraded, the applications on that machine will be stopped and the unit will @@ -84,12 +93,13 @@ outside of the cycle of new releases of Kubernetes. This includes: +- calico or other CNI +- coredns - easyrsa - etcd -- flannel, calico or other CNI Note that this may include other applications which you may have installed, such as -Elasticsearch, Prometheus, Nagios, Helm, etc. +Ceph, Docker Registry, MetalLB, Volcano, etc. @@ -117,15 +127,16 @@ juju refresh etcd --channel=1.29/stable To upgrade **etcd** itself, you will need to set the **etcd** charm channel config. To determine the correct channel, go to the -[releases section of the bundle repository][bundle-repo] page and check the relevant -**Charmed Kubernetes** bundle. Within the bundle, you should see which channel -the **etcd** charm is configured to use. +[releases section of the bundle repository](https://github.com/charmed-kubernetes/bundle/tree/main/releases) +and check the relevant **Charmed Kubernetes** bundle. Within the bundle, you will see +which channel the **etcd** charm is configured to use. Once you know the correct channel, set the **etcd** charm's channel config: ```bash juju config etcd channel=3.4/stable ``` + ### Upgrading MetalLB (if used) Previous versions of Charmed Kubernetes adopted a two charm approach for MetalLB. These were deployed in a K8s model with the suggested name `metallb-system`. @@ -171,18 +182,18 @@ juju refresh easyrsa --channel=1.29/stable ``` Any other infrastructure charms should be upgraded in a similar way. For -example, if you are using the flannel CNI: +example, if you are using the calico CNI: ```bash -juju refresh flannel --channel=1.29/stable +juju refresh calico --channel=1.29/stable ```

Note: Some services may be briefly interrupted during the upgrade process. Upgrading -your CNI (e.g. flannel) will cause a small amount of network downtime. Upgrading -easyrsa will not cause any downtime. The behaviour of other +your CNI (e.g. calico) will cause a small amount of network downtime. Upgrading +easyrsa will not cause any downtime. The behaviour of other components you have added to your cluster may vary - check individual documentation for these charms for more information on upgrades.

@@ -193,8 +204,8 @@ for these charms for more information on upgrades. Before you upgrade the **Kubernetes** components, you should be aware of the exact release you wish to upgrade to. -The **Kubernetes** charms use **snap** _channels_ to manage the version of -**Kubernetes** to use. Channels are explained in more detail in the +The **Kubernetes** charms use snap _channels_ to manage the version of +**Kubernetes** they use. Channels are explained in more detail in the [official snap documentation][snap-channels], but in terms of **Kubernetes** all you need to know are the major and minor version numbers and the 'risk-level': @@ -205,7 +216,7 @@ need to know are the major and minor version numbers and the 'risk-level': | beta | Latest alpha/beta of Kubernetes for the specified release | | edge | Nightly builds of the specified release of Kubernetes | -For most use cases, it is strongly recommended to use the 'stable' version of charms. +For most use cases, it is strongly recommended to use the _stable_ charm versions. ### Upgrading the **kube-api-loadbalancer** @@ -222,11 +233,6 @@ that of NGINX rather than Kubernetes. ### Upgrading the **kubernetes-control-plane** units -**Note**: Older versions of Charmed Kubernetes used `kubernetes-master` as the charm name. This has been updated -to `kubernetes-control-plane`. However, it is not possible to rename a deployed application. If you -originally installed version 1.23 or before, your units will follow the old naming scheme and you should -substitute `kubernetes-control-plane` for `kubernetes-master` in the following commands. - To start upgrading the Kubernetes control plane units, first upgrade the charm: ```bash diff --git a/pages/k8s/ceph.md b/pages/k8s/ceph.md index a4720c37..c27877aa 100644 --- a/pages/k8s/ceph.md +++ b/pages/k8s/ceph.md @@ -4,7 +4,7 @@ markdown_includes: nav: "kubernetes/docs/shared/_side-navigation.md" context: title: "Ceph storage" - description: How to get Ceph deployed and related to Kubernetes in order to have a default storage class. This allows for easy storage allocation. + description: How to include Ceph CSI for Ceph storage support. keywords: quickstart tags: [getting_started] sidebar: k8smain-sidebar @@ -13,39 +13,35 @@ layout: [base, ubuntu-com] toc: False --- -## How to add **Ceph** storage +Many workloads that you may want to run on your Kubernetes cluster will require some +form of available storage. This guide will help you deploy **Charmed Kubernetes** with +**Ceph** container storage support. Available storage backends include `ceph-xfs`, +`ceph-ext4`, and `cephfs`. -Many things you will want to use your Kubernetes cluster for will require some -form of available storage. Storage is quite a large topic -- this guide will -focus on just adding some quick storage using **Ceph**, so you can get up and -running quickly. +## Deploying Ceph and Charmed Kubernetes -### What you'll need +In this section, we'll create a small Ceph cluster along with a basic Charmed Kubernetes +environment that includes Ceph CSI support. -- A **Charmed Kubernetes** environment set up and running. See the [quickstart][quickstart] if you haven't . -- An existing **Ceph** cluster or the ability to create one. +### Deploy Ceph -### Deploying Ceph - -Setting up a Ceph cluster is easy with Juju. For this example we will deploy -three ceph monitor nodes: +Start by deploying three Ceph monitor nodes: ```bash - juju deploy -n 3 ceph-mon +juju deploy -n 3 ceph-mon ``` -...and then we'll add three storage nodes. For the storage nodes, we will also -specify some actual storage for these nodes to use by using `-- storage`. In -this case the Juju charm uses labels for different types of storage: +...and then add three storage nodes. For the storage nodes, we will specify additional +deployment parameters by using the `--storage` flag: ```bash - juju deploy -n 3 ceph-osd --storage osd-devices=32G,2 --storage osd-journals=8G,1 +juju deploy -n 3 ceph-osd --storage osd-devices=32G,2 --storage osd-journals=8G,1 ``` -This will deploy a storage node, and attach two 32GB devices for storage and -8GB for journalling. As we have asked for 3 machines, this means a total of -192GB of storage and 24GB of journal space. The storage comes from whatever the -default storage class is for the cloud (e.g., on AWS this will be EBS volumes). +The storage nodes above will have two 32GB devices for storage and 8GB for journaling. +As we have asked for 3 machines, this means a total of 192GB of storage and 24GB of +journal space. The storage comes from whatever the default storage class is for the +cloud (e.g., on AWS this will be EBS volumes). ```bash juju integrate ceph-osd ceph-mon @@ -55,88 +51,75 @@ juju integrate ceph-osd ceph-mon
Note:

For more on how Juju makes use of storage, please see the relevant - Juju documentation

+Juju documentation

-### Relating to Charmed Kubernetes +### Deploy Charmed Kubernetes with Ceph CSI -Making **Charmed Kubernetes** aware of your **Ceph** cluster just requires a **Juju** relation. - -```bash -juju integrate ceph-mon kubernetes-control-plane -``` - -Note that the **Ceph** CSI containers require privileged access: +Ceph CSI creates various Kubernetes resources, including pods. Therefore it requires the +`kubernetes-control-plane` application to run in privileged mode. Create a suitable environment as follows: ```bash +juju deploy charmed-kubernetes juju config kubernetes-control-plane allow-privileged=true -``` - -And finally, you need the pools that are defined in the storage class: -```bash -juju run ceph-mon/0 create-pool name=xfs-pool +juju deploy ceph-csi +juju deploy ceph-fs +juju config ceph-csi cephfs-enable=True ``` -```yaml -unit-ceph-mon-0: - id: c12f0688-f31b-4956-8314-abacd2d6516f - status: completed - timing: - completed: 2018-08-20 20:49:34 +0000 UTC - enqueued: 2018-08-20 20:49:31 +0000 UTC - started: 2018-08-20 20:49:31 +0000 UTC - unit: ceph-mon/0 -``` +Now add the relevant Ceph integrations: ```bash -juju run ceph-mon/0 create-pool name=ext4-pool -``` - -```yaml -unit-ceph-mon-0: - id: 4e82d93d-546f-441c-89e1-d36152c082f2 - status: completed - timing: - completed: 2018-08-20 20:49:45 +0000 UTC - enqueued: 2018-08-20 20:49:41 +0000 UTC - started: 2018-08-20 20:49:43 +0000 UTC - unit: ceph-mon/0 +juju integrate ceph-csi:kubernetes kubernetes-control-plane:juju-info +juju integrate ceph-csi:ceph-client ceph-mon:client +juju integrate ceph-fs:ceph-mds ceph-mon:mds ``` -### Verifying things are working +## Verify things are working -Now you can look at your **Charmed Kubernetes** cluster to verify things are -working. Running: +Check the Charmed Kubernetes cluster to verify Ceph cluster resources are +available. Running: ```bash -kubectl get sc,po +juju ssh kubernetes-control-plane/leader -- kubectl get sc,po --namespace default ``` -... should return output similar to: +...should return output similar to: ```no-highlight -NAME PROVISIONER AGE -storageclass.storage.k8s.io/ceph-ext4 csi-rbdplugin 7m -storageclass.storage.k8s.io/ceph-xfs (default) csi-rbdplugin 7m - -NAME READY STATUS RESTARTS AGE -pod/csi-rbdplugin-attacher-0 1/1 Running 0 7m -pod/csi-rbdplugin-cnh9k 2/2 Running 0 7m -pod/csi-rbdplugin-lr66m 2/2 Running 0 7m -pod/csi-rbdplugin-mnn94 2/2 Running 0 7m -pod/csi-rbdplugin-provisioner-0 1/1 Running 0 7m +NAME PROVISIONER RECLAIMPOLICY VOLUMEBINDINGMODE ALLOWVOLUMEEXPANSION AGE +storageclass.storage.k8s.io/ceph-ext4 rbd.csi.ceph.com Delete Immediate true 142m +storageclass.storage.k8s.io/ceph-xfs (default) rbd.csi.ceph.com Delete Immediate true 142m +storageclass.storage.k8s.io/cephfs cephfs.csi.ceph.com Delete Immediate true 127m + +NAME READY STATUS RESTARTS AGE +pod/csi-cephfsplugin-4gs22 3/3 Running 0 127m +pod/csi-cephfsplugin-ljfw9 3/3 Running 0 127m +pod/csi-cephfsplugin-mlbdx 3/3 Running 0 127m +pod/csi-cephfsplugin-provisioner-9f479bcc4-48v8f 5/5 Running 0 127m +pod/csi-cephfsplugin-provisioner-9f479bcc4-92bt6 5/5 Running 0 127m +pod/csi-cephfsplugin-provisioner-9f479bcc4-hbb82 5/5 Running 0 127m +pod/csi-cephfsplugin-wlp2w 3/3 Running 0 127m +pod/csi-cephfsplugin-xwdb2 3/3 Running 0 127m +pod/csi-rbdplugin-b8nk8 3/3 Running 0 142m +pod/csi-rbdplugin-bvqwn 3/3 Running 0 142m +pod/csi-rbdplugin-provisioner-85dc49c6c-9rckg 7/7 Running 0 142m +pod/csi-rbdplugin-provisioner-85dc49c6c-f6h6k 7/7 Running 0 142m +pod/csi-rbdplugin-provisioner-85dc49c6c-n47fx 7/7 Running 0 142m +pod/csi-rbdplugin-vm25h 3/3 Running 0 142m ``` -If you have installed **Helm**, you can then add a chart to verify the persistent volume is automatically created for you. +If you have installed **Helm**, you can then add a chart to verify the persistent volume +is automatically created for you: ```bash helm install stable/phpbb kubectl get pvc ``` -Which should return something similar to: +...which should return output similar to: ```ǹo-highlight NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE @@ -145,9 +128,15 @@ calling-wombat-phpbb-phpbb Bound pvc-b1d1131da4bd11e8 8Gi RWO data-calling-wombat-mariadb-0 Bound pvc-b1df7ac9a4bd11e8 8Gi RWO ceph-xfs 34s ``` -### Conclusion +## Warning: Removal -Now you have a **Ceph** cluster talking to your **Kubernetes** cluster. From +When the `ceph-csi` charm is removed, it will not clean up Ceph pools that were created +when the relation with `ceph-mon:client` was joined. If you wish to remove ceph pools, +use the `delete-pool` action on a `ceph-mon` unit. + +## Conclusion + +Now you have a **Ceph** cluster integrated with your **Charmed Kubernetes** cluster. From here you can install any of the things that require storage out of the box. @@ -160,4 +149,3 @@ here you can install any of the things that require storage out of the box.

See the guide to contributing or discuss these docs in our public Mattermost channel.

- diff --git a/pages/k8s/openstack-integration.md b/pages/k8s/openstack-integration.md index 7fb9291e..bcc5ba59 100644 --- a/pages/k8s/openstack-integration.md +++ b/pages/k8s/openstack-integration.md @@ -43,12 +43,11 @@ applications: options: allow-privileged: "true" openstack-integrator: - annotations: charm: openstack-integrator num_units: 1 trust: true openstack-cloud-controller: - charm: openstack-integrator + charm: openstack-cloud-controller cinder-csi: charm: cinder-csi relations: @@ -58,7 +57,7 @@ relations: - [openstack-cloud-controller:openstack, openstack-integrator:clients] - [easyrsa:client, cinder-csi:certificates] - [kubernetes-control-plane:kube-control, cinder-csi:kube-control] - - [openstack-integrator:clients, cinder-csi:openstack, ] + - [openstack-integrator:clients, cinder-csi:openstack] ``` To use the overlay with the **Charmed Kubernetes** bundle, specify it during deploy like this: @@ -67,7 +66,7 @@ To use the overlay with the **Charmed Kubernetes** bundle, specify it during dep juju deploy charmed-kubernetes --overlay ~/path/openstack-overlay.yaml --trust ``` -... and remember to fetch the configuration file! +...and remember to fetch the configuration file! ```bash juju ssh kubernetes-control-plane/leader -- cat config > ~/.kube/config @@ -114,7 +113,7 @@ allow for external access. #### LoadBalancer-type Pod Services -To use Octavia for `LoadBalancer` type services in the cluster, you will also need to set the +To use Octavia for `LoadBalancer`-type services in the cluster, you will need to set the `subnet-id` config to the appropriate tenant subnet where your nodes reside, and if desired, the `floating-network-id` config to whatever network you want FIPs created in. See the [Charm config docs][charm-config] for more details. @@ -134,7 +133,7 @@ You can verify that the application and replicas have been created with: kubectl get deployments hello-world ``` -Which should return output similar to: +...which should return output similar to: ```bash NAME READY UP-TO-DATE AVAILABLE AGE @@ -164,12 +163,11 @@ curl http://202.49.242.3:8080 Hello Kubernetes! ``` - #### API Server Load Balancer -If desired, the openstack-integrator can also replace kubeapi-load-balancer and create a native +If desired, the `openstack-integrator` can also replace `kubeapi-load-balancer` and create a native OpenStack load balancer for the Kubernetes API server, which simplifies the model and is properly -HA, which kubeapi-load-balancer on its own is not. To enable this, use the appropriate overlay +HA, which `kubeapi-load-balancer` on its own is not. To enable this, use the appropriate overlay ([ Versions >= 1.29][asset-openstack-lb-overlay], [Versions <= 1.28][asset-openstack-lb-overlay-1.28]): ```yaml @@ -179,12 +177,11 @@ applications: options: allow-privileged: "true" openstack-integrator: - annotations: charm: openstack-integrator num_units: 1 trust: true openstack-cloud-controller: - charm: openstack-integrator + charm: openstack-cloud-controller cinder-csi: charm: cinder-csi relations: @@ -194,7 +191,7 @@ relations: - [openstack-cloud-controller:openstack, openstack-integrator:clients] - [easyrsa:client, cinder-csi:certificates] - [kubernetes-control-plane:kube-control, cinder-csi:kube-control] - - [openstack-integrator:clients, cinder-csi:openstack, ] + - [openstack-integrator:clients, cinder-csi:openstack] - [kubernetes-control-plane:loadbalancer-external, openstack-integrator:lb-consumer] ``` @@ -204,7 +201,7 @@ in. See the [Charm config docs][charm-config] for more details. ### Using Cinder Volumes -Many pods you may wish to deploy will require storage. Although you can use any type +Many pods you may wish to deploy will require storage. Although you can use any type of storage supported by Kubernetes (see the [storage documentation][storage]), you also have the option to use Cinder storage volumes, if supported by your OpenStack. @@ -276,7 +273,7 @@ Keystone. This is covered in detail in the [Keystone and LDAP documentation][lda ### Upgrading the integrator charm -The openstack-integrator has not specifically been tied to the version of Charmed Kubernetes installed and may +The `openstack-integrator` has not specifically been tied to the version of Charmed Kubernetes installed and may generally be upgraded at any time with the following command: ```bash @@ -284,28 +281,30 @@ juju refresh openstack-integrator ``` The 1.29/stable release of `openstack-integrator` replaces the relation for using Octavia as a loadbalancer for the API Service. -The 1.29/stable release of `kubernetes-control-plane` drops the responsibility of deploying `cinder-csi` and the `openstack-controller-manager` +The 1.29/stable release of `kubernetes-control-plane` drops the responsibility of deploying `cinder-csi` and the `openstack-controller-manager`. In order to upgrade the control-plane and worker charms, follow this process: **1. Upgrade the openstack-integrator charm**: -``` +```bash juju refresh openstack-integrator --switch --channel=1.29/stable ``` -**2. Relate to the control-plane application:** +**2. Integrate the kubernetes-control-plane application:** +```bash +juju integrate openstack-integrator:lb-consumer kubernetes-control-plane:loadbalancer-external ``` -juju relate openstack-integrator:lb-consumer kubernetes-control-plane:loadbalancer-external -``` + **3. Deploy and migrate to the `openstack-cloud-controller` charm** (See its [charm docs][openstack-cloud-controller-readme] for details). **4. Deploy and migrate to the `cinder-csi` charm** (See its [charm docs][cinder-csi-readme] for details). **5. Remove the loadbalancer relation to the control-plane:** -```juju remove-relation openstack-integrator:loadbalancer kubernetes-control-plane:loadbalancer``` - +```bash +juju remove-relation openstack-integrator:loadbalancer kubernetes-control-plane:loadbalancer +``` ### Troubleshooting @@ -319,7 +318,6 @@ the log history for that specific unit: juju debug-log --replay --include openstack-integrator/0 ``` - [octavia]: https://docs.openstack.org/octavia/latest/reference/introduction.html @@ -334,7 +332,7 @@ juju debug-log --replay --include openstack-integrator/0 [cinder-csi-readme]: https://charmhub.io/cinder-csi/ [install]: /kubernetes/docs/install-manual [ldap]: /kubernetes/docs/ldap -[charm-config]: https://ubuntu.com/kubernetes/docs/charm-openstack-integrator#configuration +[charm-config]: https://charmhub.io/openstack-integrator/configure
diff --git a/pages/k8s/release-notes.md b/pages/k8s/release-notes.md index 93987be2..70e632e6 100644 --- a/pages/k8s/release-notes.md +++ b/pages/k8s/release-notes.md @@ -16,7 +16,7 @@ toc: False # 1.29 -### February 23, 2024 - `charmed-kubernetes --channel 1.29/stable` +### February 12, 2024 - `charmed-kubernetes --channel 1.29/stable` The release bundle can also be [downloaded here](https://raw.githubusercontent.com/charmed-kubernetes/bundle/main/releases/1.29/bundle.yaml). @@ -25,8 +25,8 @@ The release bundle can also be [downloaded here](https://raw.githubusercontent.c ### Charmed Operator Framework (Ops) We're pleased to announce the completion of the Charmed Kubernetes refactor that began -last year. Charms have moved from the `reactive` and `pod-spec` styles to the `ops` -framework in order to enable access to common charm libraries, gain better Juju support, +last year. Core charms have moved from the `reactive` and `pod-spec` styles to the `ops` +framework. This shift aims to enable access to common charm libraries, gain better Juju support, and provide a more consistent charming experience for community engagement. ### Out of the box monitoring enhancements @@ -39,6 +39,20 @@ observability tools. This release expands our COS integration so that it includes rich monitoring for the control plane and worker node components of Charmed Kubernetes. +### Ceph CSI + +Ceph CSI resource management has been decoupled from the `kubernetes-control-plane` +charm. All new deployments should leverage the [ceph-csi][] charm for Ceph storage +provisioning, including support for CephFS. See the [updated documentation][ceph] for +details on deploying Charmed Kubernetes with Ceph support. + +### OpenStack integration + +OpenStack capabilities (including cinder storage and cloud provider) have been decoupled +from the `kubernetes-control-plane` charm. All new deployments should leverage the new +`openstack-integrator`, `openstack-controller-manager`, and `cinder-csi` charms. See the +[updated documentation][openstack] for more details. + ### NVIDIA GPU Operator The new [nvidia-gpu-operator][] charm simplifies the management of NVIDIA GPU resources @@ -74,7 +88,29 @@ support and is offered as an alternative to the default Calico CNI. All bug fixes and other feature updates in this release can be found at [the launchpad milestone page for 1.29](https://launchpad.net/charmed-kubernetes/+milestone/1.29). -## Known Issues + + +## Notes and Known Issues + +### Integration gaps + +While Charmed Kubernetes core charms have been rewritten in the `ops` framework, some +integrations with other charms in the Juju ecosystem are not available in this release. +If your deployment depends on the following, we recommend remaining on Charmed Kubernetes +1.28 until compatible support is provided in a subsequent release: + +- AWS IAM authentication: [aws-iam](https://charmhub.io/aws-iam) +- Keystone authentication: [keystone](https://charmhub.io/keystone) +- Vault storage: [vault](https://charmhub.io/vault) + +### Cloud integrators + +Direct integration between `[aws|azure|gcp|openstack]-integrator` charms and +`kubernetes-control-plane` has been removed in this release. The recommended method for +enabling native cloud features is to use the respective out-of-tree cloud provider +charms. See the cloud-specific documentation for details. + +### Bugs A list of known bugs scheduled to be fixed in the first maintenance release can be found on the [1.29+ck1 milestone page](https://launchpad.net/charmed-kubernetes/+milestone/1.29+ck1). @@ -86,12 +122,16 @@ on the [1.29+ck1 milestone page](https://launchpad.net/charmed-kubernetes/+miles For details of other deprecation notices and API changes for Kubernetes 1.29, please see the relevant sections of the [upstream release notes][upstream-changelog-1.29]. +[upstream-changelog-1.29]: https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG/CHANGELOG-1.29.md#deprecation + [rel]: /kubernetes/docs/release-notes +[ceph-csi]: https://charmhub.io/ceph-csi?channel=1.29/stable +[ceph]: /kubernetes/docs/ceph +[openstack]: /kubernetes/openstack-integration [nvidia-gpu-operator]: https://charmhub.io/nvidia-gpu-operator?channel=1.29/stable [gpu-workers]: /kubernetes/docs/gpu-workers [install-local]: /kubernetes/docs/install-local [install-existing]: /kubernetes/docs/install-existing -[upstream-changelog-1.29]: https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG/CHANGELOG-1.29.md#deprecation diff --git a/pages/k8s/upgrading.md b/pages/k8s/upgrading.md index f438fa40..56b70d6c 100644 --- a/pages/k8s/upgrading.md +++ b/pages/k8s/upgrading.md @@ -22,29 +22,27 @@ toc: False
- Upgrade to 1.27 - Upgrade to 1.26 - Upgrade to 1.25 + Upgrade to 1.29 + Upgrade to 1.28 + Upgrade to 1.27
+It is recommended that you keep your **Kubernetes** deployment updated to the latest available stable version. You should also update the other applications which make up **Charmed Kubernetes**. Keeping up to date ensures you have the latest bug-fixes and security patches for smooth operation of your cluster. - -It is recommended that you keep your **Kubernetes** deployment updated to the latest available stable version. You should also update the other applications which make up the **Charmed Kubernetes**. Keeping up to date ensures you have the latest bug-fixes and security patches for smooth operation of your cluster. - -New minor versions of **Kubernetes** are set to release once per quarter. You can check the latest release version on the [Kubernetes release page on GitHub][k8s-release]. **Charmed Kubernetes** is kept in close sync with upstream Kubernetes: updated versions will be released within a week of a new upstream version of **Kubernetes**. +New minor versions of **Kubernetes** are set to release three times per year. You can check the latest release version on the [Kubernetes release page on GitHub][k8s-release].
Note: -

Kubernetes will automatically handle patch releases. This means that the cluster will perform an unattended automatic upgrade between patch versions, e.g. 1.23.1 to 1.23.2. Attended upgrades are only required when you wish to upgrade a minor version, e.g. 1.22.x to 1.23.x.

+

Charmed Kubernetes will automatically handle patch releases. This means that the cluster will perform an unattended automatic upgrade between patch versions, e.g. 1.29.1 to 1.29.2. Attended upgrades are only required when you wish to upgrade a minor version, e.g. 1.28.x to 1.29.x.

-You can see which version of each application is currently deployed by running +You can see which version of each application is currently deployed by running: ```bash juju status @@ -62,10 +60,10 @@ You should also make sure: - The machine from which you will perform the backup has sufficient internet access to retrieve updated software - Your cluster is running normally -- Your Juju client and controller/models are running the latest versions (see the [Juju docs][juju-controller-upgrade]) +- Your Juju client and controller/models are running the same, stable version of Juju (see the [Juju docs][juju-controller-upgrade]) - You read the [Upgrade notes][notes] to see if any caveats apply to the versions you are upgrading to/from - You read the [Release notes][release-notes] for the version you are upgrading to, which will alert you to any important changes to the operation of your cluster -- You read the [Upstream release notes](https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG/CHANGELOG-1.24.md#deprecation) for details of deprecation notices and API changes for Kubernetes 1.24 which may impact your workloads. +- You read the [Upstream release notes](https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG/CHANGELOG-1.29.md#deprecation) for details of Kubernetes deprecation notices and API changes that may impact your workloads It is also important to understand that **Charmed Kubernetes** will only upgrade and if necessary migrate, components relating specifically to elements of @@ -76,18 +74,20 @@ deprecated APIs. ## Specific upgrade instructions -We have never recommended skipping a version in an upgrade and no longer provide generic upgrade -instructions for any version of Charmed Kubernetes. Please follow the specific upgrade +We have never recommended skipping a version in an upgrade and do not provide generic upgrade +instructions for any version of **Charmed Kubernetes**. Please follow the specific upgrade documentation for the version you are upgrading to. ### Supported versions +- [Upgrading from 1.28.x to 1.29.x](/kubernetes/docs/1.29/upgrading) +- [Upgrading from 1.27.x to 1.28.x](/kubernetes/docs/1.28/upgrading) - [Upgrading from 1.26.x to 1.27.x](/kubernetes/docs/1.27/upgrading) -- [Upgrading from 1.25.x to 1.26.x](/kubernetes/docs/1.26/upgrading) -- [Upgrading from 1.24.x to 1.25.x](/kubernetes/docs/1.25/upgrading) ### Older versions +- [Upgrading from 1.25.x to 1.26.x](/kubernetes/docs/1.26/upgrading) +- [Upgrading from 1.24.x to 1.25.x](/kubernetes/docs/1.25/upgrading) - [Upgrading from 1.23.x to 1.24.x](/kubernetes/docs/1.24/upgrading) - [Upgrading from 1.22.x to 1.23.x](/kubernetes/docs/1.23/upgrading) - [Upgrading from 1.21.x to 1.22.x](/kubernetes/docs/1.22/upgrading)