Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore(ci_visibility): only import ddtrace when pytest ddtrace module is enabled #8999

Conversation

romainkomorn-exdatadog
Copy link
Contributor

@romainkomorn-exdatadog romainkomorn-exdatadog commented Apr 16, 2024

This defers importing ddtrace in the pytest, pytest-bdd, and pytest-benchmark plugins until after it's been determined that the plugin is enabled with (eg: by running pytest --ddtrace).

The fixtures that use ddtrace defer importing until the feature is actually requested due to the fact that fixtures imported later (eg: after plugins have loaded) are not usable.

Even though this doesn't actually address the issues reported in #8281 or #4845 (due to the side effects of ddtrace/__init__.py), this change is still helpful in other ways (in particular since I'll be introducing a new version of the pytest plugin shortly).

Fixtures were confirmed to still be working by:

  • using --ddtrace-patch-all
  • adding tags to spans using the ddspan fixture
  • seeing that the ddtracer fixture returns the tracer object.

Checklist

  • Change(s) are motivated and described in the PR description
  • Testing strategy is described if automated tests are not included in the PR
  • Risks are described (performance impact, potential for breakage, maintainability)
  • Change is maintainable (easy to change, telemetry, documentation)
  • Library release note guidelines are followed or label changelog/no-changelog is set
  • Documentation is included (in-code, generated user docs, public corp docs)
  • Backport labels are set (if applicable)
  • If this PR changes the public interface, I've notified @DataDog/apm-tees.

Reviewer Checklist

  • Title is accurate
  • All changes are related to the pull request's stated goal
  • Description motivates each change
  • Avoids breaking API changes
  • Testing strategy adequately addresses listed risks
  • Change is maintainable (easy to change, telemetry, documentation)
  • Release note makes sense to a user of the library
  • Author has acknowledged and discussed the performance implications of this PR as reported in the benchmarks PR comment
  • Backport labels are set in a manner that is consistent with the release branch maintenance policy

@romainkomorn-exdatadog romainkomorn-exdatadog added the changelog/no-changelog A changelog entry is not required for this PR. label Apr 16, 2024
@romainkomorn-exdatadog romainkomorn-exdatadog marked this pull request as ready for review April 17, 2024 10:00
@romainkomorn-exdatadog romainkomorn-exdatadog requested review from a team as code owners April 17, 2024 10:00
@pr-commenter
Copy link

pr-commenter bot commented Apr 17, 2024

Benchmarks

Benchmark execution time: 2024-04-18 16:08:55

Comparing candidate commit 0a47b44 in PR branch romain.komorn/CIVIS-9657/pytest_dont_import_ddtrace_by_default with baseline commit 5376eb5 in branch main.

Found 2 performance improvements and 4 performance regressions! Performance is the same for 195 metrics, 9 unstable metrics.

scenario:flasksimple-appsec-get

  • 🟩 execution_time [-229.057µs; -170.846µs] or [-3.524%; -2.628%]

scenario:flasksimple-appsec-telemetry

  • 🟥 execution_time [+238.411µs; +293.619µs] or [+3.795%; +4.674%]

scenario:httppropagationextract-datadog_tracecontext_tracestate_not_propagated_on_trace_id_no_match

  • 🟩 max_rss_usage [-1024.426KB; -951.484KB] or [-4.673%; -4.340%]

scenario:httppropagationinject-ids_only

  • 🟥 max_rss_usage [+717.295KB; +773.649KB] or [+3.394%; +3.660%]

scenario:httppropagationinject-with_dd_origin

  • 🟥 max_rss_usage [+600.449KB; +720.921KB] or [+2.834%; +3.403%]

scenario:httppropagationinject-with_tags_max_size

  • 🟥 max_rss_usage [+707.207KB; +759.570KB] or [+3.342%; +3.589%]

ddtrace/contrib/pytest/plugin.py Outdated Show resolved Hide resolved
ddtrace/contrib/pytest/_plugin_v1.py Show resolved Hide resolved
@romainkomorn-exdatadog romainkomorn-exdatadog enabled auto-merge (squash) April 19, 2024 09:45
@romainkomorn-exdatadog romainkomorn-exdatadog merged commit 01d5e01 into main Apr 19, 2024
68 checks passed
@romainkomorn-exdatadog romainkomorn-exdatadog deleted the romain.komorn/CIVIS-9657/pytest_dont_import_ddtrace_by_default branch April 19, 2024 10:04
brettlangdon pushed a commit that referenced this pull request Aug 16, 2024
…s to try-except blocks (#10196)

There are two changes in this commit:

1- remove "sub plugin" strategy introduced in #8999

The sub-plugin strategy was originally implemented to delay the import
of `ddtrace`, however that import is (in the current scenario)
inevitable because the `pyproject.toml` entry-points that tell `pytest`
where to find plugins already import `ddtrace`, and the original
motivation of the change in (#8281 and #4845) was instead addressed by
#9141 .

The motivation to revert this change is that some hooks (eg:
`pytest_load_inital_conftests`) added by the class-based plugins do not
execute when the plugin is loaded in the `pytest_configure()` hook in
the `plugin.py` file itself. All the methods in the plugin classes use
the `@staticmethod` decorator and do not benefit from being in a class.

This change allows the versioned plugin files to implement hooks by
virtue of being imported into the original plugin module.

2- move v2 functions to try-except blocks

Since CI Visibility should never affect `pytest`'s execution (or, at the
very least, should never prevent it from running to completion), the
hooks themselves should never raise exceptions. With this change, some
functions simply wrap the entire body in a try/except block, while some
others define a separate function that is then called in a try/except
block.

A decorator was not used for this purpose because some hooks yield,
while others don't, and logging relevant errors is also a little
simpler.

## Checklist
- [x] PR author has checked that all the criteria below are met
- The PR description includes an overview of the change
- The PR description articulates the motivation for the change
- The change includes tests OR the PR description describes a testing
strategy
- The PR description notes risks associated with the change, if any
- Newly-added code is easy to change
- The change follows the [library release note
guidelines](https://ddtrace.readthedocs.io/en/stable/releasenotes.html)
- The change includes or references documentation updates if necessary
- Backport labels are set (if
[applicable](https://ddtrace.readthedocs.io/en/latest/contributing.html#backporting))

## Reviewer Checklist
- [x] Reviewer has checked that all the criteria below are met 
- Title is accurate
- All changes are related to the pull request's stated goal
- Avoids breaking
[API](https://ddtrace.readthedocs.io/en/stable/versioning.html#interfaces)
changes
- Testing strategy adequately addresses listed risks
- Newly-added code is easy to change
- Release note makes sense to a user of the library
- If necessary, author has acknowledged and discussed the performance
implications of this PR as reported in the benchmarks PR comment
- Backport labels are set in a manner that is consistent with the
[release branch maintenance
policy](https://ddtrace.readthedocs.io/en/latest/contributing.html#backporting)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
changelog/no-changelog A changelog entry is not required for this PR.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants