Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

✨ Handle multiple runs of linter and coverage in metric report #223

Closed
ckunki opened this issue Jul 16, 2024 · 3 comments
Closed

✨ Handle multiple runs of linter and coverage in metric report #223

ckunki opened this issue Jul 16, 2024 · 3 comments
Assignees
Labels
feature Product feature

Comments

@ckunki
Copy link
Contributor

ckunki commented Jul 16, 2024

This ticket needs to be fixed before 2024 Nov 30, see https://github.com/actions/upload-artifact.

Currently, only a single file .lint.txt or .coverage is evaluated in metrics.py.

This also leads to problems with github actions/upload-artifact and actions/download-artifact which have slightly different semantics in @v4 vs. @v3. Additionally, actions upload- and download- must be using the same version.

The current implementation additionally is inconsistent, as the single file may be randomly selected.

The current ticket proposes to

  • to ensure unique filenames by adding a prefix for the matrix build coordinate, e.g. python version
  • enhance the report to
    • Merge the coverage of all runs for all matrix build coordinates
    • Create an aggregated version of the linter results, e.g. average, max, or min score.
  • Merging is also required when separating different classes of tests, e.g. fast vs. slow tests, see Separate long duration tests  saas-api-python#53
@ckunki ckunki added the feature Product feature label Jul 16, 2024
@tkilias
Copy link
Collaborator

tkilias commented Jul 16, 2024

In a first version, the linter reports could be merged by having a section for each matrix build coordinates

@Nicoretti
Copy link
Member

Nicoretti commented Sep 12, 2024

This issue is related to: #181

@Nicoretti
Copy link
Member

@Nicoretti Nicoretti self-assigned this Sep 30, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature Product feature
Projects
None yet
Development

No branches or pull requests

3 participants