Merge pull request #61 from fultonj/ceph_data_plane_node_set

Update Ceph docs for OpenStackDataPlaneNodeSet
openstack-k8s-operators · Sep 20, 2023 · 7fd67a3 · 7fd67a3
2 parents d93060b + fd40530
commit 7fd67a3
Show file tree

Hide file tree

Showing 3 changed files with 96 additions and 102 deletions.
diff --git a/ceph.md b/ceph.md
@@ -161,24 +161,23 @@ spec:
             mountPath: "/etc/ceph"
             readOnly: true
 ```
-The `OpenStackDataPlane` can also use `extraMounts`.
+The `OpenStackDataPlaneNodeSet` can also use `extraMounts`.
 ```yaml
 apiVersion: dataplane.openstack.org/v1beta1
-kind: OpenStackDataPlane
+kind: OpenStackDataPlaneNodeSet
 spec:
-  roles:
-    edpm-compute:
-      nodeTemplate:
-        extraMounts:
-        - extraVolType: Ceph
-          volumes:
-          - name: ceph
-            secret:
-              secretName: ceph-conf-files
-          mounts:
-          - name: ceph
-            mountPath: "/etc/ceph"
-            readOnly: true
+  ...
+  nodeTemplate:
+    extraMounts:
+    - extraVolType: Ceph
+      volumes:
+      - name: ceph
+        secret:
+          secretName: ceph-conf-files
+      mounts:
+      - name: ceph
+        mountPath: "/etc/ceph"
+        readOnly: true
 ```
 When a CR containing the above is created, an Ansible pod
 running on OpenShift mounts the files in the Ceph secret
@@ -307,14 +306,14 @@ overwrite any custom service with the same name during reconciliation.
 
 After the `ConfigMap` and `OpenStackDataPlaneService` services above
 have been created (e.g. `oc create -f ceph-nova.yaml`), update the
-`OpenStackDataPlane`
+`OpenStackDataPlaneNodeSet`
 [EDPM services list](https://openstack-k8s-operators.github.io/dataplane-operator/composable_services)
 to replace the `nova` service with `nova-custom-ceph` and add the
 `ceph-client` service.
 
 ```yaml
 apiVersion: dataplane.openstack.org/v1beta1
-kind: OpenStackDataPlane
+kind: OpenStackDataPlaneNodeSet
 spec:
   ...
   roles:
@@ -495,10 +494,10 @@ spec:
 ## Full Examples
 
 The examples above are focussed on showing how a
-single `OpenStackControlPlane` and `OpenStackDataPlane`
+single `OpenStackControlPlane` and `OpenStackDataPlaneNodeSet`
 CR can be modified to include Ceph configuration by adding
 `extraMounts` and `customServiceConfig`. Links to complete
 examples are below.
 
 - `OpenStackControlPlane`: [core_v1beta1_openstackcontrolplane_network_isolation_ceph.yaml](https://github.com/openstack-k8s-operators/openstack-operator/blob/main/config/samples/core_v1beta1_openstackcontrolplane_network_isolation_ceph.yaml)
-- `OpenStackDataPlane`: [dataplane_v1beta1_openstackdataplane_ceph.yaml](https://github.com/openstack-k8s-operators/dataplane-operator/blob/main/config/samples/dataplane_v1beta1_openstackdataplane_ceph.yaml)
+- `OpenStackDataPlaneNodeSet`: [dataplane_v1beta1_openstackdataplanenodeset_ceph.yaml](https://github.com/openstack-k8s-operators/dataplane-operator/blob/main/config/samples/dataplane_v1beta1_openstackdataplanenodeset_ceph.yaml)
diff --git a/edpm_service_overview.md b/edpm_service_overview.md
@@ -71,9 +71,9 @@ ports for the Ceph Monitoring service.
     dport: [6789, 3300]
 ```
 
-When the `run-os`
-[composable service](https://openstack-k8s-operators.github.io/dataplane-operator/composable_services/)
-runs, it executes the role
+When the `configure-os` and `run-os`
+[composable services](https://openstack-k8s-operators.github.io/dataplane-operator/composable_services/)
+run, they execute the role
 [edpm_nftables](https://github.com/openstack-k8s-operators/edpm-ansible/tree/main/roles/edpm_nftables)
 This role reads files in `/var/lib/edpm-config/firewall/`
 and creates a `edpm-rules.nft` file in `/etc/nftables/` and then
@@ -94,11 +94,11 @@ which results in the following output from the NFT command.
 		tcp dport { 3300, 6789 } ct state new counter packets 0 bytes 0 accept comment "110 allow ceph_mon"
 [root@edpm-compute-0 ~]# 
 ```
-If the service needs to be deployed after the `run-os` service has
-run, then the Ansible for that service can directly call the
-`edpm_nftables` role to update the files in `/etc/nftables` and
-reload the rules. An example of this from the `edpm_libvirt` role
-is below.
+If the service needs to be deployed after the `configure-os` and
+`run-os` services have run, then the Ansible for that service can
+directly call the `edpm_nftables` role to update the files in
+`/etc/nftables` and reload the rules. An example of this from the
+`edpm_libvirt` role is below.
 
 ```yaml
 - name: Copy qemu vnc firewall config

diff --git a/hci.md b/hci.md
@@ -11,34 +11,30 @@ deployments are also known as Hyperconverged Infrastructure (HCI).
 3. Configure OpenStack to use the collocated Ceph server
 
 In order to complete the above procedure, the `services` list of the
-`OpenStackDataPlane` CR needs to be edited.
+`OpenStackDataPlaneNodeSet` CR needs to be edited.
 
-## OpenStackDataPlane services list
+## OpenStackDataPlaneNodeSet services list
 
-EDPM nodes can be configured by creating an `OpenStackDataPlane` CRs
-(or `OpenStackDataPlaneRole` and `OpenStackDataPlaneNode` CRs)
-which the
+EDPM nodes can be configured by creating an
+`OpenStackDataPlaneNodeSet` CR which the
 [dataplane-operator](https://openstack-k8s-operators.github.io/dataplane-operator)
-will reconcile. These types of CRs have a `services` list like the
-following:
+will reconcile when an `OpenStackDataPlaneDeployment` CR is created.
+These types of CRs have a `services` list like the following:
 
 ```yaml
 apiVersion: dataplane.openstack.org/v1beta1
-kind: OpenStackDataPlane
+kind: OpenStackDataPlaneNodeSet
 spec:
   ...
-  roles:
-    edpm-compute:
-      ...
-      services:
-        - configure-network
-        - validate-network
-        - install-os
-        - configure-os
-        - run-os
-        - ovn
-        - libvirt
-        - nova
+  services:
+    - configure-network
+    - validate-network
+    - install-os
+    - configure-os
+    - run-os
+    - ovn
+    - libvirt
+    - nova
 ```
 Only the services which are on the list will be configured.
 
@@ -60,35 +56,33 @@ This example also assumes that the EDPM nodes:
 - Are at least three in number (Ceph clusters must have at least three
   nodes for redundancy)
 
-Create an `OpenStackDataPlane` CR file, e.g. `dataplane_cr.yaml` to
-represent the EDPM nodes. See
-[dataplane_v1beta1_openstackdataplane.yaml](https://github.com/openstack-k8s-operators/dataplane-operator/blob/main/config/samples/dataplane_v1beta1_openstackdataplane.yaml)
-for an example which you will modify as described in this document.
+Create an `OpenStackDataPlaneNodeSet` CR file,
+e.g. `dataplane_cr.yaml` to represent the EDPM nodes. See
+[dataplane_v1beta1_openstackdataplanenodeset.yaml](https://github.com/openstack-k8s-operators/dataplane-operator/blob/main/config/samples/dataplane_v1beta1_openstackdataplanenodeset.yaml)
+for an example to modify as described in this document.
 
 Do not yet create the CR in OpenShift as the edits described in the
-next sections will be required.
+next sections are required.
 
 ### Shorten the Service list
 
-Shorten the services list and add the `ceph-hci-pre` service so that
-it only has `configure-network`, `validate-network`, and
-`ceph-hci-pre`.
+Update the `services` list:
+
+- Add the `ceph-hci-pre` service before the `configure-os` service.
+- Remove any services after the `run-os` service for now.
 
 ```yaml
 apiVersion: dataplane.openstack.org/v1beta1
-kind: OpenStackDataPlane
+kind: OpenStackDataPlaneNodeSet
 spec:
   ...
-  roles:
-    edpm-compute:
-      ...
-      services:
-        - configure-network
-        - validate-network
-        - install-os
-        - configure-os
-        - ceph-hci-pre
-        - run-os
+  services:
+    - configure-network
+    - validate-network
+    - install-os
+    - ceph-hci-pre
+    - configure-os
+    - run-os
 ```
 In the example above the services for `ovn`, `libvirt`, and `nova`
 have been removed. If there are other services besides the one in
@@ -101,23 +95,28 @@ edpm-ansible role called `ceph-hci-pre`. This role injects a
 `ceph-networks.yaml` file into `/var/lib/edpm-config/firewall`
 so that when the `edpm_nftables` role runs, firewall ports are open
 for Ceph services. By default the `ceph-networks.yaml` file only
-contains directives to open the ports required by the Ceph RBD block
-storage service. This is because of the following default Ansible
-variable values:
+contains directives to open the ports required by the Ceph RBD
+(block), RGW (object) and NFS (files) services. This is because of the
+following default Ansible variable value:
 ```yaml
 edpm_ceph_hci_pre_enabled_services:
   - ceph_mon
   - ceph_mgr
   - ceph_osd
+  - ceph_rgw
+  - ceph_nfs
+  - ceph_rgw_frontend
+  - ceph_nfs_frontend
 ```
-If other Ceph services like RGW, CephFS, or Dashboard will be deployed
+If other Ceph services, like the Ceph Dashboard, will be deployed
 on HCI nodes, then add additional services to the enabled services
 list above. For more informatoin, see the `ceph-hci-pre` role in the
 [edpm-ansible role documentation](https://openstack-k8s-operators.github.io/edpm-ansible/roles.html).
 
-The `run-os` service is run after `ceph-hci-pre` because it enables
-the firewall rules which `ceph-hci-pre` put in place. The `run-os`
-service also configures NTP, which is requried by Ceph.
+The `configure-os` and `run-os` services are run after `ceph-hci-pre`
+because they enable the firewall rules which `ceph-hci-pre` put in
+place. The `run-os` service also configures NTP, which is requried by
+Ceph.
 
 ### Add a Ceph cluster network
 
@@ -441,21 +440,20 @@ and Ceph configuration file.
 
 ```yaml
 apiVersion: dataplane.openstack.org/v1beta1
-kind: OpenStackDataPlane
+kind: OpenStackDataPlaneNodeSet
 spec:
-  roles:
-    edpm-compute:
-      nodeTemplate:
-        extraMounts:
-        - extraVolType: Ceph
-          volumes:
-          - name: ceph
-            secret:
-              secretName: ceph-conf-files
-          mounts:
-          - name: ceph
-            mountPath: "/etc/ceph"
-            readOnly: true
+  ...
+  nodeTemplate:
+    extraMounts:
+    - extraVolType: Ceph
+      volumes:
+      - name: ceph
+        secret:
+          secretName: ceph-conf-files
+      mounts:
+      - name: ceph
+        mountPath: "/etc/ceph"
+        readOnly: true
 ```
 
 ### Restore the full services list
@@ -467,23 +465,20 @@ the full services list needs to be restored. For example:
 
 ```yaml
 apiVersion: dataplane.openstack.org/v1beta1
-kind: OpenStackDataPlane
+kind: OpenStackDataPlaneNodeSet
 spec:
   ...
-  roles:
-    edpm-compute:
-      ...
-      services:
-        - configure-network
-        - validate-network
-        - install-os
-        - configure-os
-        - ceph-hci-pre
-        - run-os
-        - ceph-client
-        - ovn
-        - libvirt
-        - nova-custom-ceph
+  services:
+    - configure-network
+    - validate-network
+    - install-os
+    - configure-os
+    - ceph-hci-pre
+    - run-os
+    - ceph-client
+    - ovn
+    - libvirt
+    - nova-custom-ceph
 ```
 In addition to restoring the default service list, the `ceph-client`
 service is added after `run-os`. The `ceph-client` service configures