From d24d82652e0658dc1201153b067887e9af2da1ca Mon Sep 17 00:00:00 2001 From: Bohdan Dobrelia Date: Fri, 29 Sep 2023 14:39:00 +0200 Subject: [PATCH] Document Nova services adoption (no extra cell) Note about remapping cell names from OSP cells naming scheme to the NG scheme with the superconductor layout. Adjust dependencies adoption (Placement, Nova cells DB) docs. Address the switch for service overrides spec instead of externalEndpoints. Remove Nova Metadata secret creation workaround from the EDPM adotopion docs. Signed-off-by: Bohdan Dobrelia --- docs/contributing/development_environment.md | 33 +++++ docs/openstack/backend_services_deployment.md | 58 ++++++++ docs/openstack/edpm_adoption.md | 15 +- docs/openstack/keystone_adoption.md | 5 +- docs/openstack/mariadb_copy.md | 17 ++- docs/openstack/nova_adoption.md | 134 ++++++++++++++++++ docs/openstack/placement_adoption.md | 8 +- 7 files changed, 244 insertions(+), 26 deletions(-) create mode 100644 docs/openstack/nova_adoption.md diff --git a/docs/contributing/development_environment.md b/docs/contributing/development_environment.md index 3af6f95ce..fcc585564 100644 --- a/docs/contributing/development_environment.md +++ b/docs/contributing/development_environment.md @@ -56,6 +56,37 @@ cd ~/install_yamls/devsetup make standalone ``` +**TODO (Q1/24)** + + # Deployment of a multi-stack topology for Nova cells v2 + + Create the edpm-compute-1 and edpm-compute-2 virtual machinee for Nova compute cells. + ``` + make edpm_compute EDPM_COMPUTE_SUFFIX=1 + make edpm_compute_repos EDPM_COMPUTE_SUFFIX=1 + ``` + + Omit the `edpm_deploy` make target to not making it managed from the control + plane running on OCP. + + Instead, deploy it as a 2nd TripleO standalone Heat stack, + with an extra OSP compute: + ``` + make standalone EDPM_COMPUTE_SUFFIX=1 EDPM_COMPUTE_CEPH_ENABLED=false + ``` + + Ssh into deployed standalone host: + ``` + ssh -i ~/install_yamls/out/edpm/ansibleee-ssh-key-id_rsa root@192.168.122.100 + ``` + Discover the deployed compute host keys (a passwordless access is required by + the Nova Live-migration feature): + ``` + ssh-copy-id -i ~/.ssh/id_rsa root@192.168.122.101 + ``` + + TODO Discover the remote compute node from the central controller node + ### Convenience steps To make our life easier we can copy the deployment passwords we'll be using @@ -116,6 +147,8 @@ export OS_CLOUD=standalone source ~/install_yamls/devsetup/scripts/edpm-deploy-instance.sh ``` +TODO(bogdando): document creating a 2nd instance on another compute cell + Confirm the image UUID can be seen in Ceph's images pool. ``` ssh -i ~/install_yamls/out/edpm/ansibleee-ssh-key-id_rsa root@192.168.122.100 sudo cephadm shell -- rbd -p images ls -l diff --git a/docs/openstack/backend_services_deployment.md b/docs/openstack/backend_services_deployment.md index 456e67a6e..cf332465e 100644 --- a/docs/openstack/backend_services_deployment.md +++ b/docs/openstack/backend_services_deployment.md @@ -62,6 +62,64 @@ podified OpenStack control plane services. ## Pre-checks +> **NOTE** The multi-cells v2 topology under adoption testinig is a central stack without multiple cells. Multi-stack topology will result +> in different cells DBs mappings and will be using a different naming scheme. + +Get the database_connection and transport_url cells mappings on the source OSP environment, before those get renamed during +the [databases adoption](mariadb_copy.md). Note those below to compare it later with the renamed post-adoption values. Post-adoption, the `nova_cell0` mapping should remain intact, while the `nova` one should get renamed to `nova_cell1`. The `nova_api` DB (not shown here) should retain its name as well. + +Ssh to the standalone node: +```bash +ssh -i ~/install_yamls/out/edpm/ansibleee-ssh-key-id_rsa root@192.168.122.100 +``` + +Firstly, check the templated view stored in the galera database of the source environment: +```bash +> podman exec -it galera-bundle-podman-0 mysql -uroot nova_api -e"select uuid,name,transport_url,database_connection,disabled from cell_mappings" | grep '|' +| uuid | name | transport_url | database_connection | disabled | +| 00000000-0000-0000-0000-000000000000 | cell0 | none:/// | mysql+pymysql://{username}:{password}@192.168.122.99/nova_cell0?read_default_file=/etc/my.cnf.d/tripleo.cnf&read_default_group=tripleo | 0 | +| 58036269-e862-4b3d-943d-2dbd0fc2d915 | default | rabbit://{username}:{password}@standalone.ctlplane.localdomain:5672/?ssl=0 | mysql+pymysql://{username}:{password}@192.168.122.99/nova?read_default_file=/etc/my.cnf.d/tripleo.cnf&read_default_group=tripleo | 0 | +``` + +Then note its Nova representation, showing names and passwords (snipped): +```bash +> podman exec -it nova_api nova-manage cell_v2 list_cells --verbose | grep '|' +| Name | UUID | Transport URL | Database Connection | Disabled | +| cell0 | 00000000-0000-0000-0000-000000000000 | none:/// | mysql+pymysql://192.168.122.99/nova_cell0?read_default_file=/etc/my.cnf.d/tripleo.cnf&read_default_group=tripleo | False | +| default | 58036269-e862-4b3d-943d-2dbd0fc2d915 | rabbit://standalone.ctlplane.localdomain:5672/?ssl=0 | mysql+pymysql://192.168.122.99/nova?read_default_file=/etc/my.cnf.d/tripleo.cnf&read_default_group=tripleo | False | +``` + +Also note the databases list on the source environment: +```bash +> podman exec -it galera-bundle-podman-0 mysql -uroot nova_api -e"show databases;" ++--------------------+ +| Database | ++--------------------+ +| cinder | +| glance | +| information_schema | +| keystone | +| mysql | +| nova | +| nova_api | +| nova_cell0 | +| ovs_neutron | +| performance_schema | +| placement | ++--------------------+ +``` +Note how the `nova`, `nova_api`, `nova_cell0` databases reside in the same DB instance of the central standalone stack. + +Finally, note the cell ID for the started earlier workload: +``` +> podman exec -it galera-bundle-podman-0 mysql -uroot nova_api -e"select cell_id from nova_api.instance_mappings;" ++---------+ +| cell_id | ++---------+ +| 2 | ++---------+ +``` + ## Procedure - backend services deployment * Make sure you are using the OpenShift namespace where you want the diff --git a/docs/openstack/edpm_adoption.md b/docs/openstack/edpm_adoption.md index 0e9ea4d70..af61df154 100644 --- a/docs/openstack/edpm_adoption.md +++ b/docs/openstack/edpm_adoption.md @@ -95,19 +95,6 @@ EOF $(cat ~/install_yamls/out/edpm/ansibleee-ssh-key-id_rsa | base64 | sed 's/^/ /') EOF ``` -* Create the Nova Metadata secret (Workaround while nova isn't adopted yet): - - ```bash - oc apply -f - < "\${dbname}".sql done - EOF ``` * Restore the databases from .sql files into the podified MariaDB: - ``` + ```bash # db schemas to rename on import declare -A db_name_map db_name_map["nova"]="nova_cell1" @@ -154,7 +153,7 @@ COLLATION=utf8_general_ci * Check that the databases were imported correctly: - ``` + ```bash oc run mariadb-client --image $MARIADB_IMAGE -i --rm --restart=Never -- \ mysql -h "${PODIFIED_MARIADB_IP}" -uroot "-p${PODIFIED_DB_ROOT_PASSWORD}" -e 'SHOW databases;' \ | grep keystone diff --git a/docs/openstack/nova_adoption.md b/docs/openstack/nova_adoption.md new file mode 100644 index 000000000..bcee8fc9f --- /dev/null +++ b/docs/openstack/nova_adoption.md @@ -0,0 +1,134 @@ +# Nova adoption + +## Prerequisites + +* Previous Adoption steps completed. Notably, + * the [service databases](mariadb_copy.md) + must already be imported into the podified MariaDB; + * the [Keystone service](keystone_adoption.md) needs to be imported; + * the [Placement service](placement_adoption.md) needs to be imported; + * the [Glance service](glance_adoption.md) needs to be imported; + * the [Neutron service](neutron_adoption.md) needs to be imported. + +## Variables + +(There are no shell variables necessary currently.) + +## Procedure - Nova adoption + +* Patch OpenStackControlPlane to deploy Nova: + + ``` + oc patch openstackcontrolplane openstack -n openstack --type=merge --patch ' + spec: + nova: + enabled: true + apiOverride: + route: {} + template: + apiServiceTemplate: + override: + service: + internal: + metadata: + annotations: + metallb.universe.tf/address-pool: internalapi + metallb.universe.tf/allow-shared-ip: internalapi + metallb.universe.tf/loadBalancerIPs: 172.17.0.80 + spec: + type: LoadBalancer + metadataServiceTemplate: + override: + service: + metadata: + annotations: + metallb.universe.tf/address-pool: internalapi + metallb.universe.tf/allow-shared-ip: internalapi + metallb.universe.tf/loadBalancerIPs: 172.17.0.80 + spec: + type: LoadBalancer + secret: osp-secret + ' + ``` + +## Post-checks + +> **NOTE** Post-adoption, there should be no compute hosts in the central (standalone) stack for +> the only supported multi-cell topology there. + +The former pre-adoption central cell's `nova` DB (in terms of OSP) is going to +be no longer used. + +FIXME: How to/should we adopt Nova compute hosts from the +central stack of the source envirnment, if any? + +Verify the other cells databases naming (should become unchanged): +``` +nova_api -> nova_api +nova_cell0 -> nova_cell0 +nova_cell1 -> nova_cell1 +nova -> no longer in use FXIME: its records should migrate to? +``` + +* See that Nova endpoints are defined and pointing to the + podified FQDNs and that Nova API responds. + + ```bash + alias openstack="oc exec -t openstackclient -- openstack" + + openstack endpoint list | grep nova + + # Without OpenStack client for Nova installed: + NOVA_PUBLIC_URL=$(openstack endpoint list -c 'Service Name' -c 'Service Type' -c URL | grep nova | grep public | awk '{ print $6; }') + curl "$NOVA_PUBLIC_URL" + + # With OpenStack client for Nova installed: + openstack server list + ``` + +* Check the cells mappings, compare it to the values [noted earlier](backend_services_deployment.md#pre-checks) + + ```bash + > oc rsh openstack-galera-0 mysql -u root --password=12345678 -e 'select * from nova_api.cell_mappings;' + Defaulted container "galera" out of: galera, mysql-bootstrap (init) + +---------------------+---------------------+----+--------------------------------------+-------+--------------------------------------------------------------------------------------------------------------+----------------------------------------------------------------+----------+ + | created_at | updated_at | id | uuid | name | transport_url | database_connection | disabled | + +---------------------+---------------------+----+--------------------------------------+-------+--------------------------------------------------------------------------------------------------------------+----------------------------------------------------------------+----------+ + | 2023-10-10 11:31:22 | 2023-10-10 11:31:23 | 1 | 00000000-0000-0000-0000-000000000000 | cell0 | rabbit:// | mysql+pymysql://nova_cell0:12345678@openstack/nova_cell0 | 0 | + | 2023-10-10 11:31:43 | NULL | 2 | 10e461ba-b9de-4545-b180-11e03915ca3f | cell1 | rabbit://default_user_WKaxHjAoXETXcRzQQX2:RVwgQpE3BvK6jnmOQ_uTVHFE2mnMJ5Sd@rabbitmq-cell1.openstack.svc:5672 | mysql+pymysql://nova_cell1:12345678@openstack-cell1/nova_cell1 | 0 | + +---------------------+---------------------+----+--------------------------------------+-------+--------------------------------------------------------------------------------------------------------------+----------------------------------------------------------------+----------+ + ``` + Note how cell1's `nova` DB became `nova_cell1`. + TODO bring the nova-manage list cells equivalent and note about the (snipped) passwords should remain unchanged (if we import them from tripleo passwords file), or re-generated. + + ``` + > oc rsh openstack-galera-0 mysql -u root --password=12345678 -e 'show databases;' + Defaulted container "galera" out of: galera, mysql-bootstrap (init) + +--------------------+ + | Database | + +--------------------+ + | cinder | + | glance | + | information_schema | + | keystone | + | mysql | + | neutron | + | nova_api | + | nova_cell0 | + | performance_schema | + | placement | + +--------------------+ + > oc rsh openstack-cell1-galera-0 mysql -u root --password=12345678 -e 'show databases;' + Defaulted container "galera" out of: galera, mysql-bootstrap (init) + +--------------------+ + | Database | + +--------------------+ + | information_schema | + | mysql | + | nova_cell1 | + | performance_schema | + +--------------------+ + ``` + Note how cells' databases has been assigned to corresponding DB instances. + + TODO: check for a VM previosely running in 'nova' cell, should be now in `nova_cell1` diff --git a/docs/openstack/placement_adoption.md b/docs/openstack/placement_adoption.md index 44abc147e..0d543f2f4 100644 --- a/docs/openstack/placement_adoption.md +++ b/docs/openstack/placement_adoption.md @@ -2,8 +2,12 @@ ## Prerequisites -* Previous Adoption steps completed. Notably, the service databases - must already be imported into the podified MariaDB. +* Previous Adoption steps completed. Notably, + * the [service databases](mariadb_copy.md) + must already be imported into the podified MariaDB. + * the [Keystone service](keystone_adoption.md) needs to be imported. + * the Memcached operator needs to be deployed (nothing to import for it from + the source environment). ## Variables