chore(ci_visibility): revert sub-plugin strategy and move v2 functions to try-except blocks

[romainkomorn-exdatadog]

There are two changes in this commit:

1- remove "sub plugin" strategy introduced in #8999

The sub-plugin strategy was originally implemented to delay the import of ddtrace, however that import is (in the current scenario) inevitable because the pyproject.toml entry-points that tell pytest where to find plugins already import ddtrace, and the original motivation of the change in (#8281 and #4845) was instead addressed by #9141 .

The motivation to revert this change is that some hooks (eg: pytest_load_inital_conftests) added by the class-based plugins do not execute when the plugin is loaded in the pytest_configure() hook in the plugin.py file itself. All the methods in the plugin classes use the @staticmethod decorator and do not benefit from being in a class.

This change allows the versioned plugin files to implement hooks by virtue of being imported into the original plugin module.

2- move v2 functions to try-except blocks

Since CI Visibility should never affect pytest's execution (or, at the very least, should never prevent it from running to completion), the hooks themselves should never raise exceptions. With this change, some functions simply wrap the entire body in a try/except block, while some others define a separate function that is then called in a try/except block.

A decorator was not used for this purpose because some hooks yield, while others don't, and logging relevant errors is also a little simpler.

Checklist

• PR author has checked that all the criteria below are met
• The PR description includes an overview of the change
• The PR description articulates the motivation for the change
• The change includes tests OR the PR description describes a testing strategy
• The PR description notes risks associated with the change, if any
• Newly-added code is easy to change
• The change follows the library release note guidelines
• The change includes or references documentation updates if necessary
• Backport labels are set (if applicable)

Reviewer Checklist

• Reviewer has checked that all the criteria below are met
• Title is accurate
• All changes are related to the pull request's stated goal
• Avoids breaking API changes
• Testing strategy adequately addresses listed risks
• Newly-added code is easy to change
• Release note makes sense to a user of the library
• If necessary, author has acknowledged and discussed the performance implications of this PR as reported in the benchmarks PR comment
• Backport labels are set in a manner that is consistent with the release branch maintenance policy

[github-actions]

CODEOWNERS have been resolved as:

ddtrace/contrib/pytest/_plugin_v1.py                                    @DataDog/ci-app-libraries
ddtrace/contrib/pytest/_plugin_v2.py                                    @DataDog/ci-app-libraries
ddtrace/contrib/pytest/plugin.py                                        @DataDog/ci-app-libraries

[datadog-dd-trace-py-rkomorn]

Datadog Report

Branch report: romain.komorn/SDTEST-225/remove_sub_plugin_and_wrap
Commit report: 69578db
Test service: dd-trace-py

❌ 2 Failed (0 Known Flaky), 176674 Passed, 1642 Skipped, 9h 54m 47.49s Total duration (29m 0.17s time saved)
⌛ 2 Performance Regressions

❌ Failed Tests (2)

• test_llmobs_huggingfacehub_llm - test_langchain_llmobs.py - Details

  Expand for error

   504 Server Error: Gateway Time-out for url: https://huggingface.co/api/models/google/flan-t5-xxl
  

• test_llmobs_huggingfacehub_llm - test_langchain_llmobs.py - Details

  Expand for error

   504 Server Error: Gateway Time-out for url: https://huggingface.co/api/models/google/flan-t5-xxl
  

⌛ Performance Regressions vs Default Branch (2)

• test_user_specified_service_v0 - test_elasticsearch.py 6.02s (+5.4s, +876%) - Details
• test_unspecified_service_v1 - test_elasticsearch.py 4.97s (+4.35s, +699%) - Details

[pr-commenter]

Benchmarks

Benchmark execution time: 2024-08-16 08:12:17

Comparing candidate commit 32c568f in PR branch romain.komorn/SDTEST-225/remove_sub_plugin_and_wrap with baseline commit 610038a in branch main.

Found 0 performance improvements and 0 performance regressions! Performance is the same for 353 metrics, 47 unstable metrics.