From 222e02e5f7070f585ae13f417b38ae7f97a61bbd Mon Sep 17 00:00:00 2001 From: Bohdan Dobrelia Date: Fri, 29 Sep 2023 14:39:00 +0200 Subject: [PATCH] Nova services adoption (no extra cell) Note about remapping cell DB names from OSP cells naming scheme to the NG scheme with the superconductor layout. Add a step to rename default cell as cell1, and to delete stale Nova services records from cell1 DB during initial databases import, to properly transition it into a superconductor layout later on. Adjust minor gaps in the dependencies adoption docs (Placement, Nova cells DB, OVN etc.) Address the switch for service overrides spec instead of externalEndpoints, where it is missing on the path to Nova adotpion. Remove Nova Metadata secret creation workarounds from the EDPM adotopion docs and test suits. Provide workaround for renaming 'default' cell's DB during adoption. Add test suits for Nova CP services adoption. Update EDPM adoption docs and tests to execute Nova compute post-FFU. Add missing nova and libvirt services for the edpm adoption tests. Verify no dataplane disruptions during the adoption and upgrade process. Verify Nova services still control pre-created VM workload after FFU/adotpion is done. Update and fix the composition of the services pre-check list to execute it before stopping services. Update and fix the composition of the list of the services to be stopped (cannot pull data from stopped services). Stop Nova services in stop_openstack_services instead of edpm_adoption (that was too late to do that). Get services topology specific configuration in pull_openstack_configuration. Add missing role for that as well. Also note about cleaning up delorean repos for tripleo standalone dev env. Signed-off-by: Bohdan Dobrelia --- docs/README.md | 2 + docs/openstack/edpm_adoption.md | 255 ++++++++++++++---- docs/openstack/keystone_adoption.md | 5 +- docs/openstack/mariadb_copy.md | 95 +++++-- docs/openstack/neutron_adoption.md | 2 +- docs/openstack/nova_adoption.md | 153 +++++++++++ docs/openstack/ovn_adoption.md | 6 + docs/openstack/placement_adoption.md | 8 +- .../openstack/pull_openstack_configuration.md | 62 ++++- docs/openstack/stop_openstack_services.md | 26 +- mkdocs.yml | 3 +- tests/playbooks/test_minimal.yaml | 1 + tests/playbooks/test_with_ceph.yaml | 1 + .../roles/dataplane_adoption/tasks/main.yaml | 61 +---- .../dataplane_adoption/tasks/nova_ffu.yaml | 192 +++++++++++++ .../mariadb_copy/tasks/env_vars_dst.yaml | 27 ++ .../mariadb_copy/tasks/env_vars_src.yaml | 8 + tests/roles/mariadb_copy/tasks/main.yaml | 48 ++-- .../mariadb_copy/templates/dump_dbs.bash | 2 +- .../mariadb_copy/templates/post_checks.bash | 38 ++- .../mariadb_copy/templates/pre_checks.bash | 8 +- .../mariadb_copy/templates/restore_dbs.bash | 9 +- tests/roles/nova_adoption/meta/main.yaml | 2 + tests/roles/nova_adoption/tasks/main.yaml | 103 +++++++ tests/roles/nova_adoption/tasks/verify.yaml | 37 +++ .../meta/main.yaml | 2 + .../tasks/main.yaml | 88 ++++++ .../stop_openstack_services/tasks/main.yaml | 18 +- 28 files changed, 1077 insertions(+), 185 deletions(-) create mode 100644 docs/openstack/nova_adoption.md create mode 100644 tests/roles/dataplane_adoption/tasks/nova_ffu.yaml create mode 100644 tests/roles/mariadb_copy/tasks/env_vars_dst.yaml create mode 100644 tests/roles/mariadb_copy/tasks/env_vars_src.yaml create mode 100644 tests/roles/nova_adoption/meta/main.yaml create mode 100644 tests/roles/nova_adoption/tasks/main.yaml create mode 100644 tests/roles/nova_adoption/tasks/verify.yaml create mode 100644 tests/roles/pull_openstack_configuration/meta/main.yaml create mode 100644 tests/roles/pull_openstack_configuration/tasks/main.yaml diff --git a/docs/README.md b/docs/README.md index e756b84f6..d5ea44aa6 100644 --- a/docs/README.md +++ b/docs/README.md @@ -29,6 +29,8 @@ Perform the actions from the sub-documents in the following order: * [Placement adoption](openstack/placement_adoption.md) +* [Nova adoption](openstack/nova_adoption.md) + * [Cinder adoption](openstack/cinder_adoption.md) * [Horizon adoption](openstack/horizon_adoption.md) diff --git a/docs/openstack/edpm_adoption.md b/docs/openstack/edpm_adoption.md index 0e9ea4d70..1b4491922 100644 --- a/docs/openstack/edpm_adoption.md +++ b/docs/openstack/edpm_adoption.md @@ -6,7 +6,15 @@ ## Variables -(There are no shell variables necessary currently.) +Define the shell variables used in the Fast-forward upgrade steps below. +The values are just illustrative, use values that are correct for your environment: + +```bash +PODIFIED_DB_ROOT_PASSWORD=$(oc get -o json secret/osp-secret | jq -r .data.DbRootPassword | base64 -d) +CONTROLLER_SSH="ssh -i ~/install_yamls/out/edpm/ansibleee-ssh-key-id_rsa root@192.168.122.100" + +alias openstack="oc exec -t openstackclient -- openstack" +``` ## Pre-checks @@ -95,55 +103,10 @@ EOF $(cat ~/install_yamls/out/edpm/ansibleee-ssh-key-id_rsa | base64 | sed 's/^/ /') EOF ``` -* Create the Nova Metadata secret (Workaround while nova isn't adopted yet): - - ```bash - oc apply -f - < **NOTE**: Additional orchestration happening around the FFU workarounds +> configuration for Nova compute EDPM service is a subject of future changes. + +* Configure pre-FFU workarounds for Nova compute EDPM services to update its version records: + + ```yaml + oc apply -f - < **NOTE** This example scenario describes a simple single-cell setup. Real +> multi-stack topology recommended for production use results in different +> cells DBs layout, and should be using different naming schemes (not covered +> here this time). + ## Prerequisites * Make sure the previous Adoption steps have been performed successfully. @@ -13,7 +18,9 @@ cluster. * Podified MariaDB and RabbitMQ are running. No other podified control plane services are running. - * OpenStack services have been stopped + * Required services specific topology [configuration collected](pull_openstack_configuration.md#get-services-topology-specific-configuration) + + * OpenStack services have been [stopped](stop_openstack_services.md) * There must be network routability between: @@ -34,44 +41,34 @@ cluster. Define the shell variables used in the steps below. The values are just illustrative, use values that are correct for your environment: -``` -MARIADB_IMAGE=quay.io/podified-antelope-centos9/openstack-mariadb:current-podified - +```bash PODIFIED_MARIADB_IP=$(oc get svc --selector "cr=mariadb-openstack" -ojsonpath='{.items[0].spec.clusterIP}') PODIFIED_CELL1_MARIADB_IP=$(oc get svc --selector "cr=mariadb-openstack-cell1" -ojsonpath='{.items[0].spec.clusterIP}') PODIFIED_DB_ROOT_PASSWORD=$(oc get -o json secret/osp-secret | jq -r .data.DbRootPassword | base64 -d) -# Replace with your environment's MariaDB IP: -SOURCE_MARIADB_IP=192.168.122.100 -SOURCE_DB_ROOT_PASSWORD=$(cat ~/tripleo-standalone-passwords.yaml | grep ' MysqlRootPassword:' | awk -F ': ' '{ print $2; }') - # The CHARACTER_SET and collation should match the source DB # if the do not then it will break foreign key relationships # for any tables that are created in the future as part of db sync CHARACTER_SET=utf8 COLLATION=utf8_general_ci +MARIADB_IMAGE=quay.io/podified-antelope-centos9/openstack-mariadb:current-podified +# Replace with your environment's MariaDB IP: +SOURCE_MARIADB_IP=192.168.122.100 +SOURCE_DB_ROOT_PASSWORD=$(cat ~/tripleo-standalone-passwords.yaml | grep ' MysqlRootPassword:' | awk -F ': ' '{ print $2; }') ``` ## Pre-checks -* Test connection to the original DB (show databases): +* Get the count of not-OK source databases: - ``` - podman run -i --rm --userns=keep-id -u $UID $MARIADB_IMAGE \ - mysql -h "$SOURCE_MARIADB_IP" -uroot "-p$SOURCE_DB_ROOT_PASSWORD" -e 'SHOW databases;' - ``` - -* Run mysqlcheck on the original DB to look for things that are not OK: - - ``` - podman run -i --rm --userns=keep-id -u $UID $MARIADB_IMAGE \ - mysqlcheck --all-databases -h $SOURCE_MARIADB_IP -u root "-p$SOURCE_DB_ROOT_PASSWORD" | grep -v OK + ```bash + test -z "$PULL_OPENSTACK_CONFIGURATION_MYSQLCHECK_NOK" ``` * Test connection to podified DBs (show databases): - ``` + ```bash oc run mariadb-client --image $MARIADB_IMAGE -i --rm --restart=Never -- \ mysql -h "$PODIFIED_MARIADB_IP" -uroot "-p$PODIFIED_DB_ROOT_PASSWORD" -e 'SHOW databases;' oc run mariadb-client --image $MARIADB_IMAGE -i --rm --restart=Never -- \ @@ -80,17 +77,25 @@ COLLATION=utf8_general_ci ## Procedure - data copy +> **NOTE**: We'll need to transition Nova services imported later on into a +> superconductor architecture. For that, delete the old service records in +> cells DBs, starting from the cell1. New records will be registered with +> different hostnames provided by the Nova service operator. All Nova +> services, except the compute agent, have no internal state, and its service +> records can be safely deleted. Also we need to rename the former `default` cell +> as `cell1`. + * Create a temporary folder to store DB dumps and make sure it's the working directory for the following steps: - ``` + ```bash mkdir ~/adoption-db cd ~/adoption-db ``` * Create a dump of the original databases: - ``` + ```bash podman run -i --rm --userns=keep-id -u $UID -v $PWD:$PWD:z,rw -w $PWD $MARIADB_IMAGE bash < **NOTE** This example scenario describes a simple single-cell setup. Real +> multi-stack topology recommended for production use results in different +> cells DBs layout, and should be using different naming schemes (not covered +> here this time). + +## Prerequisites + +* Previous Adoption steps completed. Notably, + * the [service databases](mariadb_copy.md) + must already be imported into the podified MariaDB; + * the [Keystone service](keystone_adoption.md) needs to be imported; + * the [Placement service](placement_adoption.md) needs to be imported; + * the [Glance service](glance_adoption.md) needs to be imported; + * the [OVN DB services](ovn_adoption.md) need to be imported; + * the [Neutron service](neutron_adoption.md) needs to be imported; + * Required services specific topology [configuration collected](pull_openstack_configuration.md#get-services-topology-specific-configuration); + * OpenStack services have been [stopped](stop_openstack_services.md) + +## Variables + +Define the shell variables and aliases used in the steps below. The values are +just illustrative, use values that are correct for your environment: + +```bash +alias openstack="oc exec -t openstackclient -- openstack" +``` + +## Procedure - Nova adoption + +> **NOTE**: We assume Nova Metadata deployed on the top level and not on each +> cell level, so this example imports it the same way. If the source deployment +> has a per cell metadata deployment, adjust the given below patch as needed. +> Metadata service cannot be run in `cell0`. + +* Patch OpenStackControlPlane to deploy Nova: + + ```yaml + oc patch openstackcontrolplane openstack -n openstack --type=merge --patch ' + spec: + nova: + enabled: true + apiOverride: + route: {} + template: + secret: osp-secret + apiServiceTemplate: + override: + service: + internal: + metadata: + annotations: + metallb.universe.tf/address-pool: internalapi + metallb.universe.tf/allow-shared-ip: internalapi + metallb.universe.tf/loadBalancerIPs: 172.17.0.80 + spec: + type: LoadBalancer + customServiceConfig: | + [workarounds] + disable_compute_service_check_for_ffu=true + metadataServiceTemplate: + enabled: true # deploy single nova metadata on the top level + override: + service: + metadata: + annotations: + metallb.universe.tf/address-pool: internalapi + metallb.universe.tf/allow-shared-ip: internalapi + metallb.universe.tf/loadBalancerIPs: 172.17.0.80 + spec: + type: LoadBalancer + customServiceConfig: | + [workarounds] + disable_compute_service_check_for_ffu=true + schedulerServiceTemplate: + customServiceConfig: | + [workarounds] + disable_compute_service_check_for_ffu=true + cellTemplates: + cell0: + conductorServiceTemplate: + customServiceConfig: | + [workarounds] + disable_compute_service_check_for_ffu=true + cell1: + metadataServiceTemplate: + enabled: false # enable here to run it in a cell instead + override: + service: + metadata: + annotations: + metallb.universe.tf/address-pool: internalapi + metallb.universe.tf/allow-shared-ip: internalapi + metallb.universe.tf/loadBalancerIPs: 172.17.0.80 + spec: + type: LoadBalancer + customServiceConfig: | + [workarounds] + disable_compute_service_check_for_ffu=true + conductorServiceTemplate: + customServiceConfig: | + [workarounds] + disable_compute_service_check_for_ffu=true + ' + ``` + + The local Conductor services will be started for each cell, while the superconductor runs in `cell0`. + Note that ``disable_compute_service_check_for_ffu`` is mandatory for all imported Nova services, until + the [external dataplane imported](edpm_adoption.md), and until Nova Compute services fast-forward upgraded. + +## Post-checks + +* Check that Nova endpoints are defined and pointing to the + podified FQDNs and that Nova API responds. + + ```bash + openstack endpoint list | grep nova + openstack server list + ``` + +Compare the following outputs with the topology specific configuration +[collected earlier](pull_openstack_configuration.md#get-services-topology-specific-configuration): + +* Query the superconductor for cell1 existance: + + ```bash + oc rsh nova-cell0-conductor-0 nova-manage cell_v2 list_cells | grep -F '| cell1 |' + ``` + + The expected changes to happen: + * cell1's `nova` DB and user name become `nova_cell1`. + * Default cell is renamed to `cell1` (in a multi-cell setup, it should become indexed as the last cell instead). + * RabbitMQ transport URL no longer uses `guest`. + +* Verify no Nova compute dataplane disruptions during the adoption/upgrade process: + + ```bash + $CONTROLLER_SSH sudo podman exec -it libvirt_virtqemud virsh list --all | grep 'instance-00000001 running' + ``` + + * Verify if Nova services control the existing VM instance: + + ```bash + openstack server list | grep -qF '| test | ACTIVE |' && openstack server stop test + openstack server list | grep -qF '| test | SHUTOFF |' + $CONTROLLER_SSH sudo podman exec -it libvirt_virtqemud virsh list --all | grep 'instance-00000001 shut off' + + openstack server list | grep -qF '| test | SHUTOFF |' && openstack server start test + openstack server list | grep -F '| test | ACTIVE |' + $CONTROLLER_SSH sudo podman exec -it libvirt_virtqemud virsh list --all | grep 'instance-00000001 running' + ``` + Note that in this guide, the same host acts as a controller, and also as a compute. diff --git a/docs/openstack/ovn_adoption.md b/docs/openstack/ovn_adoption.md index f695782d0..296678413 100644 --- a/docs/openstack/ovn_adoption.md +++ b/docs/openstack/ovn_adoption.md @@ -41,6 +41,12 @@ CONTROLLER_SSH="ssh -F ~/director_standalone/vagrant_ssh_config vagrant@standalo COMPUTE_SSH="ssh -F ~/director_standalone/vagrant_ssh_config vagrant@standalone" ``` +The real value of the `SOURCE_OVSDB_IP` can be get from the puppet generated configs: + +```bash +grep -rI 'ovn_[ns]b_conn' /var/lib/config-data/puppet-generated/ +``` + ## Procedure - Stop OVN northd on all original cluster controllers. diff --git a/docs/openstack/placement_adoption.md b/docs/openstack/placement_adoption.md index 44abc147e..0d543f2f4 100644 --- a/docs/openstack/placement_adoption.md +++ b/docs/openstack/placement_adoption.md @@ -2,8 +2,12 @@ ## Prerequisites -* Previous Adoption steps completed. Notably, the service databases - must already be imported into the podified MariaDB. +* Previous Adoption steps completed. Notably, + * the [service databases](mariadb_copy.md) + must already be imported into the podified MariaDB. + * the [Keystone service](keystone_adoption.md) needs to be imported. + * the Memcached operator needs to be deployed (nothing to import for it from + the source environment). ## Variables diff --git a/docs/openstack/pull_openstack_configuration.md b/docs/openstack/pull_openstack_configuration.md index 441d92b02..a5081dfe8 100644 --- a/docs/openstack/pull_openstack_configuration.md +++ b/docs/openstack/pull_openstack_configuration.md @@ -8,7 +8,7 @@ missed or misconfigured. Make sure you have pull the os-diff repository and configure according to your environment: -[Configure os-diff](planning.md#Configuration tooling) +[Configure os-diff](planning.md#configuration-tooling) ## Pull configuration from a TripleO deployment @@ -44,4 +44,62 @@ Once the ansible playbook has been run, you should have into your local director ▾ tmp/ ▾ collect_tripleo_configs/ ▾ glance/ -``` \ No newline at end of file +``` + +## Get services topology specific configuration + +Define the shell variables used in the steps below. The values are +just illustrative, use values that are correct for your environment: + +```bash +MARIADB_IMAGE=quay.io/podified-antelope-centos9/openstack-mariadb:current-podified +SOURCE_MARIADB_IP=192.168.122.100 +SOURCE_DB_ROOT_PASSWORD=$(cat ~/tripleo-standalone-passwords.yaml | grep ' MysqlRootPassword:' | awk -F ': ' '{ print $2; }') +``` + +Note the following outputs to compare it with post-adoption values later on: + +* Test connection to the original DB: + + ```bash + PULL_OPENSTACK_CONFIGURATION_DATABASES=$(podman run -i --rm --userns=keep-id -u $UID $MARIADB_IMAGE \ + mysql -h "$SOURCE_MARIADB_IP" -uroot "-p$SOURCE_DB_ROOT_PASSWORD" -e 'SHOW databases;') + ``` + Note the `nova`, `nova_api`, `nova_cell0` databases residing in the same DB host. + +* Run mysqlcheck on the original DB to look for things that are not OK: + + ```bash + PULL_OPENSTACK_CONFIGURATION_MYSQLCHECK_NOK=$(podman run -i --rm --userns=keep-id -u $UID $MARIADB_IMAGE \ + mysqlcheck --all-databases -h $SOURCE_MARIADB_IP -u root "-p$SOURCE_DB_ROOT_PASSWORD" | grep -v OK) + ``` + +* Get Nova cells mappings from database: + + ```bash + PULL_OPENSTACK_CONFIGURATION_NOVADB_MAPPED_CELLS=$(podman run -i --rm --userns=keep-id -u $UID $MARIADB_IMAGE mysql \ + -h "$SOURCE_MARIADB_IP" -uroot "-p$SOURCE_DB_ROOT_PASSWORD" nova_api -e \ + 'select uuid,name,transport_url,database_connection,disabled from cell_mappings;') + ``` + +* Get Nova instances cell_ids from database: + + ```bash + PULL_OPENSTACK_CONFIGURATION_NOVADB_INSTANCES_CELL_IDS=$(podman run -i --rm --userns=keep-id -u $UID $MARIADB_IMAGE mysql \ + -h "$SOURCE_MARIADB_IP" -uroot "-p$SOURCE_DB_ROOT_PASSWORD" nova_api -e \ + "select cell_id from nova_api.instance_mappings;") + ``` + +* Get the host names of the registered Nova compute services: + + ```bash + PULL_OPENSTACK_CONFIGURATION_NOVA_COMPUTE_HOSTNAMES=$(podman run -i --rm --userns=keep-id -u $UID $MARIADB_IMAGE mysql \ + -h "$SOURCE_MARIADB_IP" -uroot "-p$SOURCE_DB_ROOT_PASSWORD" nova_api -e \ + "select host from nova.services where services.binary='nova-compute';") + ``` + +* Get the list of mapped Nova cells: + + ```bash + PULL_OPENSTACK_CONFIGURATION_NOVAMANAGE_CELL_MAPPINGS=$($CONTROLLER_SSH sudo podman exec -it nova_api nova-manage cell_v2 list_cells) + ``` \ No newline at end of file diff --git a/docs/openstack/stop_openstack_services.md b/docs/openstack/stop_openstack_services.md index f67099cc2..9fedc93fb 100644 --- a/docs/openstack/stop_openstack_services.md +++ b/docs/openstack/stop_openstack_services.md @@ -37,6 +37,10 @@ are in the right host, for example to stop a service: . stackrc ansible -i $(which tripleo-ansible-inventory) Controller -m shell -a "sudo systemctl stop tripleo_horizon.service" -b ``` +> **NOTE** Nova computpe services in this guide are running on the same controller hosts. +> Adjust ``CONTROLLER${i}_SSH`` commands and ``ServicesToStop`` given below to your +> source environment specific topology. + ## Pre-checks We can stop OpenStack services at any moment, but we may leave things in an @@ -54,6 +58,9 @@ openstack volume backup list --all-projects -c ID -c Status |grep -E '\| .+ing \ openstack image list -c ID -c Status |grep -E '\| .+ing \|' ``` +Also collect the [topology configuration](pull_openstack_configuration.md#get-services-topology-specific-configuration), +before stopping services required to gather it live. You will need it to compare it +with the post-adoption values later on. ## Stopping control plane services @@ -64,6 +71,7 @@ there are no ongoing operations. 1- Connect to all the controller nodes. 2- Stop the services. 3- Make sure all the services are stopped. +4- Repeat steps 1-3 for compute hosts (workloads running on dataplane will not be affected) The cinder-backup service on OSP 17.1 could be running as Active-Passive under pacemaker or as Active-Active, so we'll have to check how it's running and @@ -84,7 +92,23 @@ ServicesToStop=("tripleo_horizon.service" "tripleo_glance_api.service" "tripleo_neutron_api.service" "tripleo_nova_api.service" - "tripleo_placement_api.service") + "tripleo_placement_api.service" + "tripleo_nova_api_cron.service" + "tripleo_nova_api.service" + "tripleo_nova_conductor.service" + "tripleo_nova_metadata.service" + "tripleo_nova_scheduler.service" + "tripleo_nova_vnc_proxy.service" + # Compute services on dataplane + "tripleo_nova_compute.service" + "tripleo_nova_libvirt.target" + "tripleo_nova_migration_target.service" + "tripleo_nova_virtlogd_wrapper.service" + "tripleo_nova_virtnodedevd.service" + "tripleo_nova_virtproxyd.service" + "tripleo_nova_virtqemud.service" + "tripleo_nova_virtsecretd.service" + "tripleo_nova_virtstoraged.service") PacemakerResourcesToStop=("openstack-cinder-volume" "openstack-cinder-backup") diff --git a/mkdocs.yml b/mkdocs.yml index a9c7340cd..45cbf0a18 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -29,7 +29,7 @@ nav: - OpenStack: - openstack/planning.md - openstack/backend_services_deployment.md - - oepsntack/pull_openstack_configuration.md + - openstack/pull_openstack_configuration.md - openstack/stop_openstack_services.md - openstack/mariadb_copy.md - openstack/ovn_adoption.md @@ -38,6 +38,7 @@ nav: - openstack/ceph_backend_configuration.md - openstack/glance_adoption.md - openstack/placement_adoption.md + - openstack/nova_adoption.md - openstack/cinder_adoption.md - openstack/horizon_adoption.md - openstack/edpm_adoption.md diff --git a/tests/playbooks/test_minimal.yaml b/tests/playbooks/test_minimal.yaml index 1012206d7..8d874e1a9 100644 --- a/tests/playbooks/test_minimal.yaml +++ b/tests/playbooks/test_minimal.yaml @@ -31,6 +31,7 @@ - neutron_adoption - glance_adoption - placement_adoption + - nova_adoption - cinder_adoption - horizon_adoption - dataplane_adoption diff --git a/tests/playbooks/test_with_ceph.yaml b/tests/playbooks/test_with_ceph.yaml index fe162b877..4360c9e28 100644 --- a/tests/playbooks/test_with_ceph.yaml +++ b/tests/playbooks/test_with_ceph.yaml @@ -34,6 +34,7 @@ - ceph_backend_configuration - glance_adoption - placement_adoption + - nova_adoption - cinder_adoption - horizon_adoption - dataplane_adoption diff --git a/tests/roles/dataplane_adoption/tasks/main.yaml b/tests/roles/dataplane_adoption/tasks/main.yaml index e3c09deb6..75277710b 100644 --- a/tests/roles/dataplane_adoption/tasks/main.yaml +++ b/tests/roles/dataplane_adoption/tasks/main.yaml @@ -108,61 +108,6 @@ ssh-privatekey: "{{ edpm_encoded_privatekey|default(edpm_privatekey.content) }}" EOF -- name: "[hack] create nova-metadata-neutron-config-secret.yaml" - no_log: "{{ use_no_log }}" - ansible.builtin.shell: | - {{ shell_header }} - {{ oc_header }} - oc apply -f - <Antelope FFU + ansible.builtin.include_tasks: + file: nova_ffu.yaml diff --git a/tests/roles/dataplane_adoption/tasks/nova_ffu.yaml b/tests/roles/dataplane_adoption/tasks/nova_ffu.yaml new file mode 100644 index 000000000..e4f80cde7 --- /dev/null +++ b/tests/roles/dataplane_adoption/tasks/nova_ffu.yaml @@ -0,0 +1,192 @@ +- name: get dst database service environment variables + ansible.builtin.include_role: + name: mariadb_copy + tasks_from: env_vars_dst.yaml + +- name: configure pre-FFU workarounds for Nova compute EDPM services to update its version records + ansible.builtin.shell: | + {{ shell_header }} + {{ oc_header }} + oc apply -f - <antelope FFU + ansible.builtin.shell: | + {{ shell_header }} + {{ oc_header }} + oc exec -it nova-cell0-conductor-0 -- nova-manage db online_data_migrations + oc exec -it nova-cell1-conductor-0 -- nova-manage db online_data_migrations + register: nova_exec_result + until: nova_exec_result is success + retries: 10 + delay: 6 + +- name: Adopted Nova FFU post-checks + ansible.builtin.include_role: + name: nova_adoption + tasks_from: verify.yaml diff --git a/tests/roles/mariadb_copy/tasks/env_vars_dst.yaml b/tests/roles/mariadb_copy/tasks/env_vars_dst.yaml new file mode 100644 index 000000000..fa7d423df --- /dev/null +++ b/tests/roles/mariadb_copy/tasks/env_vars_dst.yaml @@ -0,0 +1,27 @@ +- name: get podified MariaDB service cluster IP + ansible.builtin.shell: | + {{ shell_header }} + {{ oc_header }} + oc get svc --selector "cr=mariadb-openstack" -ojsonpath='{.items[0].spec.clusterIP}' + register: podified_mariadb_ip_result + +- name: get podified cell1 MariaDB IP + ansible.builtin.shell: | + {{ shell_header }} + {{ oc_header }} + oc get svc --selector "cr=mariadb-openstack-cell1" -ojsonpath='{.items[0].spec.clusterIP}' + register: podified_cell1_mariadb_ip_result + +- name: set MariaDB copy shell vars + no_log: "{{ use_no_log }}" + ansible.builtin.set_fact: + mariadb_copy_shell_vars_dst: | + PODIFIED_MARIADB_IP={{ podified_mariadb_ip_result.stdout }} + PODIFIED_CELL1_MARIADB_IP={{podified_cell1_mariadb_ip_result.stdout}} + PODIFIED_DB_ROOT_PASSWORD="{{ podified_db_root_password }}" + + # The CHARACTER_SET and collation should match the source DB + # if the do not then it will break foreign key relationships + # for any tables that are created in the future as part of db sync + CHARACTER_SET=utf8 + COLLATION=utf8_general_ci diff --git a/tests/roles/mariadb_copy/tasks/env_vars_src.yaml b/tests/roles/mariadb_copy/tasks/env_vars_src.yaml new file mode 100644 index 000000000..66bbd5312 --- /dev/null +++ b/tests/roles/mariadb_copy/tasks/env_vars_src.yaml @@ -0,0 +1,8 @@ +- name: set src MariaDB copy shell vars + no_log: "{{ use_no_log }}" + ansible.builtin.set_fact: + mariadb_copy_shell_vars_src: | + MARIADB_IMAGE=quay.io/podified-antelope-centos9/openstack-mariadb:current-podified + # TODO: remove the default(external_...) when CI is transitioned to use 'source_...' + SOURCE_MARIADB_IP={{ source_mariadb_ip|default(external_mariadb_ip) }} + SOURCE_DB_ROOT_PASSWORD="{{ source_db_root_password|default(external_db_root_password) }}" diff --git a/tests/roles/mariadb_copy/tasks/main.yaml b/tests/roles/mariadb_copy/tasks/main.yaml index f284c3fd6..ad9155aed 100644 --- a/tests/roles/mariadb_copy/tasks/main.yaml +++ b/tests/roles/mariadb_copy/tasks/main.yaml @@ -1,38 +1,19 @@ -- name: get podified MariaDB service cluster IP - ansible.builtin.shell: | - {{ shell_header }} - {{ oc_header }} - oc get svc --selector "cr=mariadb-openstack" -ojsonpath='{.items[0].spec.clusterIP}' - register: podified_mariadb_ip_result +- name: get the source database service environment variables + ansible.builtin.include_tasks: + file: env_vars_src.yaml -- name: get podified cell1 MariaDB IP - ansible.builtin.shell: | - {{ shell_header }} - {{ oc_header }} - oc get svc --selector "cr=mariadb-openstack-cell1" -ojsonpath='{.items[0].spec.clusterIP}' - register: podified_cell1_mariadb_ip_result +- name: get the destination database service environment variables + ansible.builtin.include_tasks: + file: env_vars_dst.yaml -- name: set MariaDB copy shell vars +- name: Get the count of not-OK source databases no_log: "{{ use_no_log }}" - ansible.builtin.set_fact: - mariadb_copy_shell_vars: | - MARIADB_IMAGE=quay.io/podified-antelope-centos9/openstack-mariadb:current-podified - - PODIFIED_MARIADB_IP={{ podified_mariadb_ip_result.stdout }} - PODIFIED_CELL1_MARIADB_IP={{podified_cell1_mariadb_ip_result.stdout}} - PODIFIED_DB_ROOT_PASSWORD="{{ podified_db_root_password }}" - - # TODO: remove the default(external_...) when CI is transitioned to use 'source_...' - SOURCE_MARIADB_IP={{ source_mariadb_ip|default(external_mariadb_ip) }} - SOURCE_DB_ROOT_PASSWORD="{{ source_db_root_password|default(external_db_root_password) }}" - - # The CHARACTER_SET and collation should match the source DB - # if the do not then it will break foreign key relationships - # for any tables that are created in the future as part of db sync - CHARACTER_SET=utf8 - COLLATION=utf8_general_ci + environment: + PULL_OPENSTACK_CONFIGURATION_MYSQLCHECK_NOK: "{{ pull_openstack_configuration_mysqlcheck_nok }}" + ansible.builtin.shell: + cmd: test -z "$PULL_OPENSTACK_CONFIGURATION_MYSQLCHECK_NOK" -- name: MariaDB copy pre checks +- name: test connection to podified DBs (show databases) no_log: "{{ use_no_log }}" ansible.builtin.shell: cmd: "{{ lookup('ansible.builtin.template', 'pre_checks.bash') }}" @@ -49,5 +30,10 @@ - name: MariaDB post-checks no_log: "{{ use_no_log }}" + environment: + PULL_OPENSTACK_CONFIGURATION_DATABASES: "{{ pull_openstack_configuration_databases }}" + PULL_OPENSTACK_CONFIGURATION_NOVADB_MAPPED_CELLS: "{{ pull_openstack_configuration_novadb_mapped_cells }}" + PULL_OPENSTACK_CONFIGURATION_NOVA_COMPUTE_HOSTNAMES: "{{ pull_openstack_configuration_nova_compute_hostnames }}" + PULL_OPENSTACK_CONFIGURATION_NOVADB_INSTANCES_CELL_IDS: "{{ pull_openstack_configuration_novadb_instances_cell_ids }}" ansible.builtin.shell: cmd: "{{ lookup('ansible.builtin.template', 'post_checks.bash') }}" diff --git a/tests/roles/mariadb_copy/templates/dump_dbs.bash b/tests/roles/mariadb_copy/templates/dump_dbs.bash index 3c73ed20d..45266d03f 100644 --- a/tests/roles/mariadb_copy/templates/dump_dbs.bash +++ b/tests/roles/mariadb_copy/templates/dump_dbs.bash @@ -1,7 +1,7 @@ #!/bin/bash {{ shell_header }} {{ oc_header }} -{{ mariadb_copy_shell_vars }} +{{ mariadb_copy_shell_vars_src }} mkdir -p {{ mariadb_copy_tmp_dir }} cd {{ mariadb_copy_tmp_dir }} diff --git a/tests/roles/mariadb_copy/templates/post_checks.bash b/tests/roles/mariadb_copy/templates/post_checks.bash index dbe92c043..a68cb48c0 100644 --- a/tests/roles/mariadb_copy/templates/post_checks.bash +++ b/tests/roles/mariadb_copy/templates/post_checks.bash @@ -1,14 +1,44 @@ {{ shell_header }} {{ oc_header }} -{{ mariadb_copy_shell_vars }} +{{ mariadb_copy_shell_vars_src }} +{{ mariadb_copy_shell_vars_dst }} oc run mariadb-client --image $MARIADB_IMAGE -i --rm --restart=Never -- \ mysql -h "${PODIFIED_MARIADB_IP}" -uroot "-p${PODIFIED_DB_ROOT_PASSWORD}" -e 'SHOW databases;' \ - | grep keystone + | grep -q keystone + # ensure neutron db is renamed from ovs_neutron oc run mariadb-client --image $MARIADB_IMAGE -i --rm --restart=Never -- \ mysql -h "${PODIFIED_MARIADB_IP}" -uroot "-p${PODIFIED_DB_ROOT_PASSWORD}" -e 'SHOW databases;' \ - | grep neutron + | grep -q neutron +echo $PULL_OPENSTACK_CONFIGURATION_DATABASES | grep ' ovs_neutron ' + # ensure nova cell1 db is extracted to a separate db server and renamed from nova to nova_cell1 oc run mariadb-client --image $MARIADB_IMAGE -i --rm --restart=Never -- \ mysql -h "${PODIFIED_CELL1_MARIADB_IP}" -uroot "-p${PODIFIED_DB_ROOT_PASSWORD}" -e 'SHOW databases;' \ - | grep nova_cell1 \ No newline at end of file + | grep -q nova_cell1 + +# ensure default cell renamed to cell1, and the cell UUIDs retained intact +NOVADB_MAPPED_CELLS=$(oc rsh mariadb-openstack mysql -uroot "-p$PODIFIED_DB_ROOT_PASSWORD" \ + nova_api -e 'select uuid,name,transport_url,database_connection,disabled from cell_mappings;') +uuidf='\S{8,}-\S{4,}-\S{4,}-\S{4,}-\S{12,}' +left_behind=$(comm -23 \ + <(echo $PULL_OPENSTACK_CONFIGURATION_NOVADB_MAPPED_CELLS | grep -oE " $uuidf \S+") \ + <(echo $NOVADB_MAPPED_CELLS | tr -s "| " " " | grep -oE " $uuidf \S+")) +changed=$(comm -13 \ + <(echo $PULL_OPENSTACK_CONFIGURATION_NOVADB_MAPPED_CELLS | grep -oE " $uuidf \S+") \ + <(echo $NOVADB_MAPPED_CELLS | tr -s "| " " " | grep -oE " $uuidf \S+")) +test $(grep -Ec ' \S+$' <<<$left_behind) -eq 1 +default=$(grep -E ' default$' <<<$left_behind) +test $(grep -Ec ' \S+$' <<<$changed) -eq 1 +grep -qE " $(awk '{print $1}' <<<$default) cell1$" <<<$changed + +# ensure the registered Nova compute service name has not changed +oc rsh mariadb-openstack-cell1 mysql -uroot "-p$PODIFIED_DB_ROOT_PASSWORD" nova_cell1 \ + -e "select host from services where services.binary='nova-compute';" \ + | grep -qF '| standalone.localdomain |' +echo $PULL_OPENSTACK_CONFIGURATION_NOVA_COMPUTE_HOSTNAMES | grep -qF ' standalone.localdomain' + +# ensure the VM instance cell ID has not changed +oc rsh mariadb-openstack mysql -uroot "-p$PODIFIED_DB_ROOT_PASSWORD" nova_api -e \ + "select cell_id from instance_mappings;" | grep -qF -m1 ' 2 |' +echo $PULL_OPENSTACK_CONFIGURATION_NOVADB_INSTANCES_CELL_IDS | grep -q -m1 ' 2$' \ No newline at end of file diff --git a/tests/roles/mariadb_copy/templates/pre_checks.bash b/tests/roles/mariadb_copy/templates/pre_checks.bash index 9f61b135c..3c4d62afc 100644 --- a/tests/roles/mariadb_copy/templates/pre_checks.bash +++ b/tests/roles/mariadb_copy/templates/pre_checks.bash @@ -1,14 +1,8 @@ #!/bin/bash {{ shell_header }} {{ oc_header }} -{{ mariadb_copy_shell_vars }} +{{ mariadb_copy_shell_vars_dst }} -# Test connection to the original DB (show databases) -podman run -i --rm --userns=keep-id -u $UID -v $PWD:$PWD:z,rw -w $PWD $MARIADB_IMAGE \ - mysql -h "$SOURCE_MARIADB_IP" -uroot "-p$SOURCE_DB_ROOT_PASSWORD" -e 'SHOW databases;' -# Run mysqlcheck on the original DB to look for things that are not OK -podman run -i --rm --userns=keep-id -u $UID -v $PWD:$PWD:z,rw -w $PWD $MARIADB_IMAGE \ - mysqlcheck --all-databases -h $SOURCE_MARIADB_IP -u root "-p$SOURCE_DB_ROOT_PASSWORD" # Test connection to podified DBs (show databases) oc run mariadb-client --image $MARIADB_IMAGE -i --rm --restart=Never -- \ mysql -h "$PODIFIED_MARIADB_IP" -uroot "-p$PODIFIED_DB_ROOT_PASSWORD" -e 'SHOW databases;' diff --git a/tests/roles/mariadb_copy/templates/restore_dbs.bash b/tests/roles/mariadb_copy/templates/restore_dbs.bash index 5cea23223..062dd548f 100755 --- a/tests/roles/mariadb_copy/templates/restore_dbs.bash +++ b/tests/roles/mariadb_copy/templates/restore_dbs.bash @@ -1,7 +1,8 @@ #!/bin/bash {{ shell_header }} {{ oc_header }} -{{ mariadb_copy_shell_vars }} +{{ mariadb_copy_shell_vars_src }} +{{ mariadb_copy_shell_vars_dst }} cd {{ mariadb_copy_tmp_dir }} # db schemas to rename on import @@ -44,4 +45,8 @@ EOF container_name=$(echo "mariadb-client-${db_name}-restore" | sed 's/_/-/g') oc run ${container_name} --image ${MARIADB_IMAGE} -i --rm --restart=Never -- \ mysql -h "${db_server}" -uroot "-p${db_password}" "${db_name}" < "${db_file}" -done \ No newline at end of file +done +oc exec -it mariadb-openstack -- mysql --user=root --password=${db_server_password_map["default"]} -e \ + "update nova_api.cell_mappings set name='cell1' where name='default';" +oc exec -it mariadb-openstack-cell1 -- mysql --user=root --password=${db_server_password_map["default"]} -e \ + "delete from nova_cell1.services where host not like '%nova-cell1-%' and services.binary != 'nova-compute';" diff --git a/tests/roles/nova_adoption/meta/main.yaml b/tests/roles/nova_adoption/meta/main.yaml new file mode 100644 index 000000000..610f184fb --- /dev/null +++ b/tests/roles/nova_adoption/meta/main.yaml @@ -0,0 +1,2 @@ +dependencies: + - role: common_defaults diff --git a/tests/roles/nova_adoption/tasks/main.yaml b/tests/roles/nova_adoption/tasks/main.yaml new file mode 100644 index 000000000..a371690c9 --- /dev/null +++ b/tests/roles/nova_adoption/tasks/main.yaml @@ -0,0 +1,103 @@ +- name: deploy podified Nova conductors, scheduler, metadata, and API + ansible.builtin.shell: | + {{ shell_header }} + {{ oc_header }} + oc patch openstackcontrolplane openstack -n openstack --type=merge --patch ' + spec: + nova: + enabled: true + apiOverride: + route: {} + template: + secret: osp-secret + apiServiceTemplate: + override: + service: + internal: + metadata: + annotations: + metallb.universe.tf/address-pool: internalapi + metallb.universe.tf/allow-shared-ip: internalapi + metallb.universe.tf/loadBalancerIPs: 172.17.0.80 + spec: + type: LoadBalancer + customServiceConfig: | + [workarounds] + disable_compute_service_check_for_ffu=true + metadataServiceTemplate: + enabled: true # deploy single nova metadata on the top level + override: + service: + metadata: + annotations: + metallb.universe.tf/address-pool: internalapi + metallb.universe.tf/allow-shared-ip: internalapi + metallb.universe.tf/loadBalancerIPs: 172.17.0.80 + spec: + type: LoadBalancer + customServiceConfig: | + [workarounds] + disable_compute_service_check_for_ffu=true + schedulerServiceTemplate: + customServiceConfig: | + [workarounds] + disable_compute_service_check_for_ffu=true + cellTemplates: + cell0: + conductorServiceTemplate: + customServiceConfig: | + [workarounds] + disable_compute_service_check_for_ffu=true + cell1: + metadataServiceTemplate: + enabled: false # enable here to run it in a cell instead + override: + service: + metadata: + annotations: + metallb.universe.tf/address-pool: internalapi + metallb.universe.tf/allow-shared-ip: internalapi + metallb.universe.tf/loadBalancerIPs: 172.17.0.80 + spec: + type: LoadBalancer + customServiceConfig: | + [workarounds] + disable_compute_service_check_for_ffu=true + conductorServiceTemplate: + customServiceConfig: | + [workarounds] + disable_compute_service_check_for_ffu=true + ' + +- name: wait for Nova to start up + ansible.builtin.shell: | + {{ shell_header }} + {{ oc_header }} + oc get pod --selector=service=nova-scheduler,statefulset.kubernetes.io/pod-name=nova-scheduler-0 -o jsonpath='{.items[0].status.phase}{"\n"}' | grep Running + oc get pod --selector=service=nova-api,statefulset.kubernetes.io/pod-name=nova-api-0 -o jsonpath='{.items[0].status.phase}{"\n"}' | grep Running + oc get pod --selector=service=nova-metadata,statefulset.kubernetes.io/pod-name=nova-metadata-0 -o jsonpath='{.items[0].status.phase}{"\n"}' | grep Running + oc get pod --selector=service=nova-novncproxy,cell=cell1 -o jsonpath='{.items[0].status.phase}{"\n"}' | grep Running + oc get pod --selector=service=nova-conductor,cell=cell0 -o jsonpath='{.items[0].status.phase}{"\n"}' | grep Running + oc get pod --selector=service=nova-conductor,cell=cell1 -o jsonpath='{.items[0].status.phase}{"\n"}' | grep Running + register: nova_running_result + until: nova_running_result is success + retries: 180 + delay: 2 + +# TODO(bogdando): implement podified FQDNs check for Nova endpoints +- name: Check that Nova endpoints are defined and pointing to the podified FQDNs and that Nova API responds + ansible.builtin.shell: | + {{ shell_header }} + {{ oc_header }} + alias openstack="oc exec -t openstackclient -- openstack" + + ${BASH_ALIASES[openstack]} endpoint list | grep nova + ${BASH_ALIASES[openstack]} server list + register: nova_responding_result + until: nova_responding_result is success + retries: 60 + delay: 2 + +- name: Nova adoption post-checks + ansible.builtin.include_tasks: + file: verify.yaml diff --git a/tests/roles/nova_adoption/tasks/verify.yaml b/tests/roles/nova_adoption/tasks/verify.yaml new file mode 100644 index 000000000..58f212ce8 --- /dev/null +++ b/tests/roles/nova_adoption/tasks/verify.yaml @@ -0,0 +1,37 @@ +- name: set Nova services shell vars + no_log: "{{ use_no_log }}" + ansible.builtin.set_fact: + nova_header: | + alias openstack="oc exec -t openstackclient -- openstack" + CONTROLLER_SSH="{{ controller1_ssh }}" + +# TODO(bogdando): provide automated checks for 'The expected changes to happen' +- name: query the superconductor for cell1 existance + ansible.builtin.shell: | + {{ shell_header }} + {{ oc_header }} + oc rsh nova-cell0-conductor-0 nova-manage cell_v2 list_cells | grep -F '| cell1 |' + +- name: verify no Nova compute dataplane disruptions during the adoption/upgrade process + ansible.builtin.shell: | + {{ shell_header }} + {{ oc_header }} + {{ nova_header }} + $CONTROLLER_SSH sudo podman exec -it libvirt_virtqemud virsh list --all | grep 'instance-00000001 running' + +- name: verify if Nova services control the existing VM instance + ansible.builtin.shell: | + {{ shell_header }} + {{ oc_header }} + {{ nova_header }} + ${BASH_ALIASES[openstack]} server list | grep -qF '| test | ACTIVE |' && ${BASH_ALIASES[openstack]} server stop test + ${BASH_ALIASES[openstack]} server list | grep -qF '| test | SHUTOFF |' + $CONTROLLER_SSH sudo podman exec -it libvirt_virtqemud virsh list --all | grep 'instance-00000001 shut off' + + ${BASH_ALIASES[openstack]} server list | grep -qF '| test | SHUTOFF |' && ${BASH_ALIASES[openstack]} server start test + ${BASH_ALIASES[openstack]} server list | grep -F '| test | ACTIVE |' + $CONTROLLER_SSH sudo podman exec -it libvirt_virtqemud virsh list --all | grep 'instance-00000001 running' + register: nova_verify_result + until: nova_verify_result is success + retries: 10 + delay: 6 diff --git a/tests/roles/pull_openstack_configuration/meta/main.yaml b/tests/roles/pull_openstack_configuration/meta/main.yaml new file mode 100644 index 000000000..610f184fb --- /dev/null +++ b/tests/roles/pull_openstack_configuration/meta/main.yaml @@ -0,0 +1,2 @@ +dependencies: + - role: common_defaults diff --git a/tests/roles/pull_openstack_configuration/tasks/main.yaml b/tests/roles/pull_openstack_configuration/tasks/main.yaml new file mode 100644 index 000000000..8af393f85 --- /dev/null +++ b/tests/roles/pull_openstack_configuration/tasks/main.yaml @@ -0,0 +1,88 @@ +- name: set shell vars pull openstack configuration ssh commands + no_log: "{{ use_no_log }}" + ansible.builtin.set_fact: + pull_openstack_configuration_ssh_shell_vars: | + CONTROLLER_SSH="{{ controller1_ssh }}" + +- name: get src database service environment variables + ansible.builtin.include_role: + name: mariadb_copy + tasks_from: env_vars_src.yaml + +# NOTE(bogdando): env variables must be used to keep this consistent with documentation, +# where the stored values need to be compared with post-adoption ones w/o using ansible specifics +- name: test connection to the original DB + no_log: "{{ use_no_log }}" + ansible.builtin.shell: | + {{ shell_header }} + {{ oc_header }} + {{ mariadb_copy_shell_vars_src }} + PULL_OPENSTACK_CONFIGURATION_DATABASES=$(podman run -i --rm --userns=keep-id -u $UID $MARIADB_IMAGE \ + mysql -h "$SOURCE_MARIADB_IP" -uroot "-p$SOURCE_DB_ROOT_PASSWORD" -e 'SHOW databases;') + echo $PULL_OPENSTACK_CONFIGURATION_DATABASES + register: _databases_check + +- name: run mysqlcheck on the original DB to look for things that are not OK + no_log: "{{ use_no_log }}" + ansible.builtin.shell: | + {{ shell_header }} + {{ oc_header }} + {{ mariadb_copy_shell_vars_src }} + PULL_OPENSTACK_CONFIGURATION_MYSQLCHECK_NOK=$(podman run -i --rm --userns=keep-id -u $UID $MARIADB_IMAGE \ + mysqlcheck --all-databases -h $SOURCE_MARIADB_IP -u root "-p$SOURCE_DB_ROOT_PASSWORD" | grep -v OK) + echo $PULL_OPENSTACK_CONFIGURATION_MYSQLCHECK_NOK + register: _mysqlnok_check + +- name: get source Nova services topology specific configuration + no_log: "{{ use_no_log }}" + block: + - name: get Nova cells mappings from database + ansible.builtin.shell: | + {{ shell_header }} + {{ oc_header }} + {{ mariadb_copy_shell_vars_src }} + PULL_OPENSTACK_CONFIGURATION_NOVADB_MAPPED_CELLS=$(podman run -i --rm --userns=keep-id -u $UID $MARIADB_IMAGE mysql \ + -h "$SOURCE_MARIADB_IP" -uroot "-p$SOURCE_DB_ROOT_PASSWORD" nova_api -e \ + 'select uuid,name,transport_url,database_connection,disabled from cell_mappings;') + echo $PULL_OPENSTACK_CONFIGURATION_NOVADB_MAPPED_CELLS + register: _novadb_mapped_cells_check + + - name: get Nova instances cell_ids from database + ansible.builtin.shell: | + {{ shell_header }} + {{ oc_header }} + {{ mariadb_copy_shell_vars_src }} + PULL_OPENSTACK_CONFIGURATION_NOVADB_INSTANCES_CELL_IDS=$(podman run -i --rm --userns=keep-id -u $UID $MARIADB_IMAGE mysql \ + -h "$SOURCE_MARIADB_IP" -uroot "-p$SOURCE_DB_ROOT_PASSWORD" nova_api -e \ + "select cell_id from nova_api.instance_mappings;") + echo $PULL_OPENSTACK_CONFIGURATION_NOVADB_INSTANCES_CELL_IDS + register: _novadb_instances_cell_ids_check + + - name: get the host names of the registered Nova compute services + ansible.builtin.shell: | + {{ shell_header }} + {{ oc_header }} + {{ mariadb_copy_shell_vars_src }} + PULL_OPENSTACK_CONFIGURATION_NOVA_COMPUTE_HOSTNAMES=$(podman run -i --rm --userns=keep-id -u $UID $MARIADB_IMAGE mysql \ + -h "$SOURCE_MARIADB_IP" -uroot "-p$SOURCE_DB_ROOT_PASSWORD" nova_api -e \ + "select host from nova.services where services.binary='nova-compute';") + echo $PULL_OPENSTACK_CONFIGURATION_NOVA_COMPUTE_HOSTNAMES + register: _nova_compute_hostnames_check + + - name: get the list of mapped Nova cells + ansible.builtin.shell: | + {{ shell_header }} + {{ oc_header }} + {{ pull_openstack_configuration_ssh_shell_vars }} + PULL_OPENSTACK_CONFIGURATION_NOVAMANAGE_CELL_MAPPINGS=$($CONTROLLER_SSH sudo podman exec -it nova_api nova-manage cell_v2 list_cells) + echo $PULL_OPENSTACK_CONFIGURATION_NOVAMANAGE_CELL_MAPPINGS + register: _nova_cell_mappings_check + +- name: set facts for pulled openstack services configuration + ansible.builtin.set_fact: + pull_openstack_configuration_databases: "{{ _databases_check.stdout }}" + pull_openstack_configuration_mysqlcheck_nok: " {{ _mysqlnok_check.stdout }}" + pull_openstack_configuration_novadb_mapped_cells: "{{ _novadb_mapped_cells_check.stdout }}" + pull_openstack_configuration_novadb_instances_cell_ids: "{{ _novadb_instances_cell_ids_check.stdout }}" + pull_openstack_configuration_nova_compute_hostnames: "{{ _nova_compute_hostnames_check.stdout }}" + pull_openstack_configuration_novamanage_cell_mappings: "{{ _nova_cell_mappings_check.stdout }}" diff --git a/tests/roles/stop_openstack_services/tasks/main.yaml b/tests/roles/stop_openstack_services/tasks/main.yaml index 75b986a48..569427621 100644 --- a/tests/roles/stop_openstack_services/tasks/main.yaml +++ b/tests/roles/stop_openstack_services/tasks/main.yaml @@ -6,6 +6,7 @@ CONTROLLER2_SSH="{{ controller2_ssh }}" CONTROLLER3_SSH="{{ controller3_ssh }}" +# TODO(bogdando): split Compute services on dataplane after the multi-node/multi-cells adoption supported - name: stop control plane services no_log: "{{ use_no_log }}" ansible.builtin.shell: | @@ -22,7 +23,22 @@ "tripleo_glance_api.service" "tripleo_neutron_api.service" "tripleo_nova_api.service" - "tripleo_placement_api.service") + "tripleo_placement_api.service" + "tripleo_nova_api_cron.service" + "tripleo_nova_api.service" + "tripleo_nova_conductor.service" + "tripleo_nova_metadata.service" + "tripleo_nova_scheduler.service" + "tripleo_nova_vnc_proxy.service" + "tripleo_nova_compute.service" + "tripleo_nova_libvirt.target" + "tripleo_nova_migration_target.service" + "tripleo_nova_virtlogd_wrapper.service" + "tripleo_nova_virtnodedevd.service" + "tripleo_nova_virtproxyd.service" + "tripleo_nova_virtqemud.service" + "tripleo_nova_virtsecretd.service" + "tripleo_nova_virtstoraged.service") PacemakerResourcesToStop=("openstack-cinder-volume" "openstack-cinder-backup")