Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix #4183: Adjust Overall Performance runtime in HTML test report #4300

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

Vaibhavsahu2810
Copy link
Contributor

Description

This pull request addresses issue #4183, where the Overall Performance runtime displayed in the Nightwatch HTML report differed from the Suite runtime for single-suite runs. The discrepancy arose due to differing calculation methods for these runtime metrics, causing confusion for users analyzing test execution data.

Changes Made

Updated the aggregateStats Method:
The method now calculates the Overall Performance runtime by summing the actual execution times (timeMs) of test cases instead of relying solely on start and end timestamps.

Proof of Work

The report shows consistent runtimes across sections for both Overall Performance and Suite runtime.
image

  • Before marking your PR for review, please test and verify your changes by making appropriate modifications to any of the Nightwatch example tests (present in examples/tests directory of the project) and running them. ecosia.js and duckDuckGo.js are good examples to work with.
  • Create a new branch from master (e.g. features/my-new-feature or issue/123-my-bugfix);
  • If you're fixing a bug also create an issue if one doesn't exist yet;
  • If it's a new feature explain why do you think it's necessary. Please check with the maintainers beforehand to make sure it is something that we will accept. Usually we only accept new features if we feel that they will benefit the entire community;
  • Please avoid sending PRs which contain drastic or low level changes. If you are certain that the changes are needed, please discuss them beforehand and indicate what the impact will be;
  • If your change is based on existing functionality please consider refactoring first. Pull requests that duplicate code will most likely be ignored;
  • Do not include changes that are not related to the issue at hand;
  • Follow the same coding style with regards to spaces, semicolons, variable naming etc.;
  • Always add unit tests - PRs without tests are most of the times ignored.

Comment on lines -125 to -126
const startTime = new Date(this.results.startTimestamp).getTime();
const endTime = new Date(this.results.endTimestamp).getTime();
Copy link
Member

@garg3133 garg3133 Nov 18, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Vaibhavsahu2810 Thanks for this PR. But, were you able to figure out why the subtraction of the above two variables is giving the wrong time?

Ideally, we should try to find the root cause of the problem and try to tackle that instead of just introducing a fix. If the this.results.endTimestamp is wrong above, we should fix that because it might be causing problems at other places as well.

Also, if the tests are running in parallel, we wouldn't just want to add up the time of all the tests but instead show the time the entire test run actually took from the user's perspective (if I'm not missing something here?).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants