Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Test Markers as Test Fields #141

Merged
merged 4 commits into from
Dec 19, 2023
Merged

Conversation

jyejare
Copy link
Member

@jyejare jyejare commented Dec 6, 2023

This PR allows Betelgeuse to :

  • Add test markers as test fields while importing the test cases to Polarion

By:

  • Read test and class decorators and parse them to filter only markers
  • New functions to read test module level markers using ast methods
  • The custom config now supports the dynamic list of ignorable markers.
  • Filter out some decorators / markers which need not to be in the markers list

Helps in:

  • Filtering the tests in polarion using markers
  • Generate reports based on markers like e2e, sanity, tiers etc

Copy link
Collaborator

@elyezer elyezer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since new functionality is being added it would be great to keep the test coverage on a good shape and have tests updated to consider the new code.

I am sending this as a comment since I am not really involved on the project as I used to be.

Other than the suggestion of dropping an non-related change and adding tests I believe it is good to go.

@@ -423,7 +423,7 @@ def get_requirement_field_values(config, requirement):

def update_testcase_fields(config, testcase):
"""Apply testcase fields default values and transformations."""
if testcase.docstring and not type(testcase.docstring) == str:
if testcase.docstring and not isinstance(testcase.docstring, str):
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Any particular reason that this is needed? I feel this should not be part of the PR since we are trying to introduce pytest markers support and not change how the docstring is parsed.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@elyezer Not needed as part of PR, but the checks are not passing without this change due to flake8 rules! Also there is no harm in doing this change :)

@jyejare
Copy link
Member Author

jyejare commented Dec 11, 2023

@elyezer Added unit tests to improve the coverage and replied to your comment saying its necessary!


def parse_markers(all_markers=None):
"""Parse the markers."""
ignore_list = ['parametrize', 'skipif', 'usefixtures', 'skip_if_not_set']
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

some of these seem to be robottelo-specific.
Also, since you wish to ignore skipif marker, I'd say you'd like to also ignore rhel_ver_match, one etc.

My point here is - perhaps the ignore_list might be implemented as a CLI option rather than a static, hardcoded list?

It would make it much more flexible and easy to add more over time.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@rplevka Your wish is granted - Now the ignore list is configurable through betelgeuse_config module 7b15cbf

@jyejare jyejare force-pushed the markers_as_fields branch 5 times, most recently from c8f4378 to 2371099 Compare December 13, 2023 17:09
@SatelliteQE SatelliteQE deleted a comment from codecov-commenter Dec 14, 2023
betelgeuse/parser.py Outdated Show resolved Hide resolved
betelgeuse/parser.py Outdated Show resolved Hide resolved
betelgeuse/parser.py Show resolved Hide resolved
betelgeuse/parser.py Outdated Show resolved Hide resolved
betelgeuse/parser.py Outdated Show resolved Hide resolved
@codecov-commenter
Copy link

Codecov Report

All modified and coverable lines are covered by tests ✅

Comparison is base (371feb4) 91.46% compared to head (e7c4c73) 92.12%.

Additional details and impacted files
@@            Coverage Diff             @@
##           master     #141      +/-   ##
==========================================
+ Coverage   91.46%   92.12%   +0.65%     
==========================================
  Files           6        6              
  Lines         879      914      +35     
==========================================
+ Hits          804      842      +38     
+ Misses         75       72       -3     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

betelgeuse/parser.py Outdated Show resolved Hide resolved
Copy link
Member

@rplevka rplevka left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ACK,
thanks for addressing my comments

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants