Merge qe_common role #123

ayefimov-1 · 2024-07-11T21:53:06Z

Merge QE Common role into master

elfiesmelfie

I fixed the linter errors already, but there are still a few more changes that should be made

elfiesmelfie · 2024-07-12T14:30:33Z

roles/qe_common/tasks/endpoint_tests.yml

+- name: Get Endpoint
+  ansible.builtin.shell:
+    cmd: |
+      oc project openstack     


oc rsh openstackclient ... does the same as kubectl exec.
You don't need to do -- to separate the command
You can pass the -n flag to tell it which namespace to run in

elfiesmelfie · 2024-07-12T14:31:14Z

roles/qe_common/tasks/cred_tests.yml

+    cmd: |
+      oc get crd "{{ item }}"
+  register: output
+  failed_when: output.rc != 0


This failure condition is implicit for the shell module, so it's usually omitted.

elfiesmelfie · 2024-07-12T14:33:25Z

roles/qe_common/tasks/main.yml

@@ -0,0 +1,64 @@
+---
+- when: container_list is defined
+  name: "Verify container - {{ container_polar_id}}"


Please change the name of this var to container_test_id, rather than container_polar_id, since it's more intuitive, rather than needing to know that polar is short for Polarian and what Polarian is.

Also, please add in defaults for these values in roles/qe_common/defaults/main.yml.

This is so that the test ID is optional, since you might want to use this for some pre-checks, without having a test_id.

elfiesmelfie · 2024-07-12T14:37:31Z

roles/qe_common/tasks/main.yml

+  loop: "{{ proj_list }}"
+
+
+-   when: 


Suggested change

- when:

- when:

elfiesmelfie · 2024-07-12T14:46:41Z

.ansible-lint

@@ -2,6 +2,8 @@
 exclude_paths:
    - ci/
    - roles/telemetry_autoscaling
+    - roles/telemetry_logging
+    - roles/qe_common


Please try not to skip linting this role.

I ran ansible-lint locally, and found some errors that prevent the roles from running

ansible-lint is really useful for catching small, hard to see errors in syntax or layout, as well as being able to provide some additional feedback that keeps the content easier to grok and maintain

FYI: If you un-skip the "command-instead-of-module" check in .ansible-lint, it gives you suggestions for what modules could be used instead of shell for certain commands

elfiesmelfie · 2024-07-12T14:49:03Z

.ansible-lint

@@ -2,6 +2,8 @@
 exclude_paths:
    - ci/
    - roles/telemetry_autoscaling
+    - roles/telemetry_logging
+    - roles/qe_common


Suggested change

- roles/qe_common

elfiesmelfie · 2024-07-12T14:52:32Z

roles/qe_common/tasks/main.yml

@@ -0,0 +1,64 @@
+---
+- when: container_list is defined
+  name: "Verify container - {{ container_polar_id}}"


https://ansible.readthedocs.io/projects/lint/rules/jinja/

jinja[spacing]: Jinja2 spacing could be improved: Verify container - {{ container_polar_id}} -> Verify container - {{ container_polar_id }} (warning)

elfiesmelfie · 2024-07-12T14:54:13Z

roles/qe_common/tasks/main.yml

+- when: node_list is defined
+  name: "Verify OSP node - {{ node_polar_id }}"
+  ansible.builtin.include_tasks: node_tests.yml
+  loop: "{{ node_list }}"


key-order[task]: You can improve the task key order to: name, when, ansible.builtin.include_tasks, loop
roles/qe_common/tasks/main.yml:32 Task/Handler: Verify OSP node - {{ node_polar_id }}
https://ansible.readthedocs.io/projects/lint/rules/key-order/

(Same for the other tasks in this file

roles/qe_common/README.md

softwarefactory-project-zuul · 2024-07-12T15:14:47Z

Build failed (check pipeline). Post recheck (without leading slash)
to rerun all jobs. Make sure the failure cause has been resolved before
you rerun jobs.

https://review.rdoproject.org/zuul/buildset/838cb4e2e4644f569ca1f922fbe31a82

❌ openstack-k8s-operators-content-provider FAILURE in 6m 06s
⚠️ functional-tests-on-osp18 SKIPPED Skipped due to failed job openstack-k8s-operators-content-provider

elfiesmelfie

It looks okay for now. This role is being called anywhere yet, so let's merge it, and we can resolve issues that come up later one this is hooked up to testing

mgirgisf

Its seems that it doesn't have any obvious syntax error to me, its okay to merged we are not triggering it yet.
Thanks Alex it contains a variety of common tasks that can be used in different other tests.

softwarefactory-project-zuul · 2024-07-17T16:10:38Z

Build failed (check pipeline). Post recheck (without leading slash)
to rerun all jobs. Make sure the failure cause has been resolved before
you rerun jobs.

https://review.rdoproject.org/zuul/buildset/2fed2e1b296143c698fc94427414a526

✔️ feature-verification-tests-noop SUCCESS in 5s
✔️ openstack-k8s-operators-content-provider SUCCESS in 1h 53m 37s
❌ functional-tests-on-osp18 FAILURE in 1h 37m 21s

elfiesmelfie · 2024-07-23T15:18:19Z

Build failed (check pipeline). Post recheck (without leading slash) to rerun all jobs. Make sure the failure cause has been resolved before you rerun jobs.

https://review.rdoproject.org/zuul/buildset/2fed2e1b296143c698fc94427414a526

✔️ feature-verification-tests-noop SUCCESS in 5s ✔️ openstack-k8s-operators-content-provider SUCCESS in 1h 53m 37s ❌ functional-tests-on-osp18 FAILURE in 1h 37m 21s

This failure is unrelated to this change.
I synced with master which will trigger a new buildset

elfiesmelfie · 2024-07-23T15:19:35Z

roles/common/README.md

+
+  For subscription_tests.yml tasks: 
+
+     subscription_polar_id


These var names need to be updated to match the updated var names (*_test_id)

elfiesmelfie · 2024-07-23T15:21:10Z

roles/common/README.md

+Example Playbook
+----------------
+
+Typically, for this role the tests should *not* use a "main.yml" and import or include all the tests in the role. On the contrary, a tests should explicitly include specific tests needed for a given job.


This can be removed.

The behaviour has changed to work by setting appropriate vars to select the tests that run. The vars are set and then used by the role on import.

Ideally, the vars would have the <role_name>_ prefix to them

elfiesmelfie · 2024-07-23T15:23:16Z

roles/common/README.md

+  hosts: controller
+  gather_facts: no
+  vars:
+     proj_out_file: verify_logging_projects_exist_lresults.log


The project_out_file is not mentioned elsewhere.

elfiesmelfie · 2024-07-26T17:28:15Z

roles/common/tasks/endpoint_tests.yml

+      oc project openstack     
+      kubectl exec openstackclient -- openstack endpoint list --service="{{ item[0] }}" --service="{{ item[1] }}"  --interface="{{ item[2] }}"


You can use oc rsh openstackclient openstack endpoint list ....

the -n openstack should be included to specify the project/namespace

What happens if the endpoint doesn't exist? Does the openstackclient return a non-zero return code?

roles/common/tasks/container_tests.yml

elfiesmelfie · 2024-07-26T17:42:18Z

roles/common/tasks/container_tests.yml

+- name: Get container status
+  ansible.builtin.shell:
+    cmd: |
+      podman ps -a --format "{{ '{{.Names}} {{.Status}}' }}" | grep "{{ item }}" | awk '{print $2;}'


For readability, I recommend using something instead of item.

You can use a useful variable name, and tell the loop in main to use that same var name for its loop var.

The syntax would be similar to: https://docs.ansible.com/ansible/latest/playbook_guide/playbooks_loops.html#stacking-loops-via-include-tasks, but without the nested loops.
I would suggest container_name as the loop var here, so that reading this task is easier.

Here's an example playbook:

--- - hosts: localhost tasks: - ansible.builtin.debug: msg: "{{ item }}" loop: - "first" - "second" - ansible.builtin.debug: msg: "{{ my_var }}" loop: - "third" - "fourth" loop_control: loop_var: my_var

New readme file for role

new playbook

duplicate file

added service test file

added for easy of use

fixed typo

removed redundant results file generation

remove redundant results file generation

remove redundant results file generation.

remove redundant results file generation

updated test_id var name

updated role name

moved role to new dir

initial file

qe_common role name change to common

test id var change

remove qe_common role dir

roles/qe_common name change to roles/common

qe_common role rename to common

remove unneeded line

lint changes

elfiesmelfie · 2024-08-07T17:34:05Z

.ansible-lint

+    - roles/telemetry_logging
+    - roles/commmon


Suggested change

- roles/telemetry_logging

- roles/commmon

elfiesmelfie · 2024-08-07T17:35:12Z

roles/common/README.md

+  hosts: controller
+  gather_facts: no
+  vars:
+     proj_out_file: verify_logging_projects_exist_lresults.log


Suggested change

proj_out_file: verify_logging_projects_exist_lresults.log

elfiesmelfie · 2024-08-07T17:37:23Z

roles/common/README.md

+
+  For subscription_tests.yml tasks: 
+
+     subscription_polar_id


Suggested change

subscription_polar_id

subscription_test_id

elfiesmelfie · 2024-08-07T17:37:37Z

roles/common/README.md

+Example Playbook
+----------------
+
+Typically, for this role the tests should *not* use a "main.yml" and import or include all the tests in the role. On the contrary, a tests should explicitly include specific tests needed for a given job.


Suggested change

Typically, for this role the tests should *not* use a "main.yml" and import or include all the tests in the role. On the contrary, a tests should explicitly include specific tests needed for a given job.

elfiesmelfie · 2024-08-07T17:38:50Z

roles/telemetry_logging/README.md

@@ -0,0 +1,53 @@
+telemetry_logging
+=========


Suggested change

=========

=================

elfiesmelfie · 2024-08-07T17:39:08Z

roles/telemetry_logging/README.md

+  For journal_tests.yml
+
+    identifiers_test_id
+      - polarion id for test


Pleas update this description

elfiesmelfie · 2024-08-07T17:44:18Z

roles/telemetry_logging/tasks/journal_tests.yml

+  ansible.builtin.shell:
+    cmd:
+      tstamp=$(date -d '30 minute ago' "+%Y-%m-%d %H:%M:%S")
+      journalctl -t "{{ item }}" --no-pager -S "${tstamp}" | wc -l


You could convert this to a command if you want, by removing the use of wc and making use of journal_output.stdout_lines to count the number of returned lines. This would also make debugging easier, since using verbose output would let you see what was returned from the journalctl command, rather than just seeing a number.

The failed_when condition could become ``journal_wc.stdout_lines | length <= 1

softwarefactory-project-zuul · 2024-08-14T19:39:55Z

Build failed (check pipeline). Post recheck (without leading slash)
to rerun all jobs. Make sure the failure cause has been resolved before
you rerun jobs.

https://softwarefactory-project.io/zuul/t/rdoproject.org/buildset/6fb59fd84b444f8e82a02eaf41d8cc6a

✔️ feature-verification-tests-noop SUCCESS in 4s
❌ openstack-k8s-operators-content-provider FAILURE in 12m 52s
⚠️ functional-tests-on-osp18 SKIPPED Skipped due to failed job openstack-k8s-operators-content-provider

elfiesmelfie · 2024-08-07T17:45:39Z

roles/common/README.md

@@ -0,0 +1,119 @@
+common
+=========


Suggested change

=========

======

elfiesmelfie · 2024-08-19T19:25:59Z

roles/common/tasks/cred_tests.yml

+    cmd: |
+      oc get crd "{{ item }}"
+  changed_when: false
+  register: output


Is there a fail condition for this? Does it need one?

elfiesmelfie · 2024-08-19T19:28:46Z

roles/common/tasks/endpoint_tests.yml

+      oc project openstack     
+      kubectl exec openstackclient -- openstack endpoint list --service="{{ item[0] }}" --service="{{ item[1] }}"  --interface="{{ item[2] }}"


What happens if the endpoint doesn't exist? Does the openstackclient return a non-zero return code?

elfiesmelfie · 2024-08-19T19:47:55Z

roles/common/tasks/node_tests.yml

I don't believe these tests are valid.
Zuul doesn't know or care that the nodes are VMs, and the executer, which run the playbooks for the job, does not have these VMs running.
We pass nodesets into the jobs, but these are for zuul to give to nodepool, which provides the test servers.
In the case of the Zuul instance on rdo, nodepool talks to various cloud providers, which then provision VMs, based on the nodeset labels.
The only information that Zuul has/needs about the hosts (crc, computes, controller, etc) is the ansible inv file, which has the hostname, IP address, etc that zuul needs to run the playbooks that we pass to it.

roles/common/tasks/main.yml

elfiesmelfie · 2024-08-19T20:02:46Z

roles/common/tasks/proj_tests.yml

+- name: Verify Project exists - "{{ item }}"
+  ansible.builtin.shell: 
+    cmd: |
+      oc project "{{ item }}"


This will switch to a different project. This may cause unexpected behaviour of subsequent commands are run without a --namespace/-n argument.
A better check would be

oc project list | grep "{{ item}}"

(please verify that this command is correct)

A more efficient approach might be to get the project list and then loop through the output to make sure that all the expected project are there.

softwarefactory-project-zuul · 2024-08-19T22:41:15Z

Build failed (check pipeline). Post recheck (without leading slash)
to rerun all jobs. Make sure the failure cause has been resolved before
you rerun jobs.

https://softwarefactory-project.io/zuul/t/rdoproject.org/buildset/74a17349039442d8a6732bc2f65be016

✔️ feature-verification-tests-noop SUCCESS in 4s
✔️ openstack-k8s-operators-content-provider SUCCESS in 2h 29m 56s
❌ functional-tests-on-osp18 FAILURE in 1h 42m 13s

elfiesmelfie · 2024-08-22T14:28:37Z

roles/common/tasks/subscription_tests.yml

+- name: Verify subscription
+  ansible.builtin.shell:
+    cmd: |
+      oc get subscriptions -n "{{ subscription_nspace }}" "{{ item }}" 


IMHO, this should have a similar format to the endpoint tests, so that a single loop can cover multiple namespaces.

elfiesmelfie · 2024-09-17T12:21:31Z

roles/common/tasks/pod_tests.yml

+- name: Get Pod Instance "{{ pod_status_str }}" 
+  ansible.builtin.shell:
+    cmd: |
+      oc get pods -n  "{{ pod_nspace }}" | grep "{{ item }}" | grep "{{ pod_status_str }}" | awk '{print $1;}'


If this fails, there is no indication until the next task.
There is also no helpful information to see what might have caused the error.

The next task will fail if there is nothing returned from here.

Suggested change

oc get pods -n "{{ pod_nspace }}" | grep "{{ item }}" | grep "{{ pod_status_str }}" | awk '{print $1;}'

oc get pods -n "{{ pod_nspace }}" | grep "{{ item }}" | grep "{{ pod_status_str }}" | awk '{print $1;}'

failed_when:

- podinstance.stdout_lines | length == 0

elfiesmelfie · 2024-09-17T12:22:32Z

roles/common/tasks/pod_tests.yml

+- name: Check terminated pod 
+  ansible.builtin.shell:
+    cmd: | 
+      oc get pod -n "{{ pod_nspace }} {{ podinstance.stdout }}"


Suggested change

oc get pod -n "{{ pod_nspace }} {{ podinstance.stdout }}"

oc get pod -n "{{ pod_nspace }}" "{{ podinstance.stdout }}"

Please don't apply these suggestions, they are reflective of what is going on in PR#149

Add service tests to the logging job adding the tasks from #123 into a spearate PR

ayefimov-1 requested review from elfiesmelfie and mgirgisf July 11, 2024 21:53

elfiesmelfie requested changes Jul 12, 2024

View reviewed changes

ayefimov-1 requested a review from elfiesmelfie July 12, 2024 18:16

elfiesmelfie mentioned this pull request Jul 15, 2024

Merge Zuul Logging job #113

Draft

elfiesmelfie approved these changes Jul 16, 2024

View reviewed changes

mgirgisf approved these changes Jul 17, 2024

View reviewed changes

elfiesmelfie reviewed Jul 26, 2024

View reviewed changes

ayefimov-1 added 19 commits August 2, 2024 14:47

Create README.md

8c98ef2

New readme file for role

Create container_tests.yml

46a758f

new playbook

Adding new playbooks to this testing role

71c049f

Delete roles/qe_common/tasks/credential_tests.yml

cdae659

duplicate file

Create service_tests.yml

3dc4358

added service test file

Create main.yml

245c2a1

added for easy of use

Update main.yml

f72fd4d

fixed typo

Update main.yml

2bf3caa

Update container_tests.yml

76c49d2

removed redundant results file generation

Update cred_tests.yml

37fa0d9

remove redundant results file generation

Update container_tests.yml

da302a4

remove redundant results file generation.

Update endpoint_tests.yml

02e3e31

remove redundant results file generation.

Update file_tests.yml

904235b

remove redundant results file generation.

Update manifest_tests.yml

f15fcb8

remove redundant results file generation.

Update node_tests.yml

9cdb01e

remove redundant results file generation

Update pod_tests.yml

f024c30

remove redundant results file generation

Update proj_tests.yml

3e339eb

remove redundant results file generation

Update service_tests.yml

66c3cae

Update subscription_tests.yml

822132c

remove redundant results file generation

ayefimov-1 added 16 commits August 2, 2024 14:47

Update main.yml

e325278

updated test_id var name

Update README.md

dbce856

updated role name

Create README.md

9b55e78

Update README.md

5ec4c85

moved role to new dir

Create main.yml

3bda671

initial file

role name change

86bc4c4

qe_common role name change to common

Update README.md

cc8994b

test id var change

Delete roles/qe_common directory

606fc82

remove qe_common role dir

Update .ansible-lint

7e50ef4

roles/qe_common name change to roles/common

Update README.md

ffb00b3

qe_common role rename to common

Update cred_tests.yml

d47e091

remove unneeded line

Update endpoint_tests.yml

39c2ff1

remove unneeded line

Update proj_tests.yml

705a2cd

remove unneeded line

Update service_tests.yml

9b3451a

remove unneeded line

Update subscription_tests.yml

f8c40ad

remove unneeded line

Update README.md

6e3283c

lint changes

elfiesmelfie force-pushed the alexy_logging2 branch from 7f6cb87 to 6e3283c Compare August 2, 2024 13:47

Add telemetry_logging role (#124) (#138)

44884f5

elfiesmelfie requested changes Aug 7, 2024

View reviewed changes

elfiesmelfie added 2 commits August 7, 2024 18:45

Merge branch 'master' into alexy_logging2

5db5aba

Fix syntax error: change_when -> changed_when

ac94272

elfiesmelfie requested changes Aug 19, 2024

View reviewed changes

[common] Rename var: nspace -> pod_nspace, service_nspace

a8891d4

elfiesmelfie reviewed Aug 22, 2024

View reviewed changes

elfiesmelfie reviewed Sep 17, 2024

View reviewed changes

myadla added a commit that referenced this pull request Sep 27, 2024

OSPRH-9659

a728912

Add service tests to the logging job adding the tasks from #123 into a spearate PR

myadla mentioned this pull request Sep 27, 2024

Add service tests to the logging job #153

Draft

myadla mentioned this pull request Dec 10, 2024

Add service and file tests to the logging job #200

Merged

		oc project openstack
		kubectl exec openstackclient -- openstack endpoint list --service="{{ item[0] }}" --service="{{ item[1] }}" --interface="{{ item[2] }}"

	oc get pod -n "{{ pod_nspace }} {{ podinstance.stdout }}"
	oc get pod -n "{{ pod_nspace }}" "{{ podinstance.stdout }}"

Merge qe_common role #123

Are you sure you want to change the base?

Merge qe_common role #123

Conversation

ayefimov-1 commented Jul 11, 2024

elfiesmelfie left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

softwarefactory-project-zuul bot commented Jul 12, 2024

elfiesmelfie left a comment

Choose a reason for hiding this comment

mgirgisf left a comment

Choose a reason for hiding this comment

softwarefactory-project-zuul bot commented Jul 17, 2024

elfiesmelfie commented Jul 23, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

softwarefactory-project-zuul bot commented Aug 14, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

softwarefactory-project-zuul bot commented Aug 19, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment