Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add benchmark workflow (RPi4 / Arm Cortex-A72) #69

Merged
merged 16 commits into from
Jun 20, 2024

Conversation

potsrevennil
Copy link
Contributor

@potsrevennil potsrevennil commented Jun 19, 2024

This PR depends on #68

I Added a benchmark workflow. Note that the workflow will only run on main branch or PR labeled as benchmark. The tests script is also refactored a bit in order to show the benchmark summary on GitHub action.

I refactored the nix shell specification in order to speed up ci, and also
disabled ci jobs running on self-host runner when pushed to fork repositories by the way.

@potsrevennil potsrevennil force-pushed the bench-ci branch 4 times, most recently from b4e1126 to 4e78bdd Compare June 19, 2024 08:20
@mkannwischer mkannwischer changed the title Add benchmark workflow Add benchmark workflow (RPi4 / Arm Cortex-A72) Jun 19, 2024
@potsrevennil potsrevennil marked this pull request as ready for review June 19, 2024 09:00
@potsrevennil potsrevennil requested a review from a team June 19, 2024 09:00
@potsrevennil potsrevennil force-pushed the bench-ci branch 4 times, most recently from 4762b78 to 3d754dd Compare June 19, 2024 10:45
@potsrevennil potsrevennil added the benchmark this PR should be benchmarked in CI label Jun 19, 2024
@potsrevennil potsrevennil marked this pull request as draft June 19, 2024 10:46
@potsrevennil potsrevennil marked this pull request as ready for review June 20, 2024 02:50
Makefile Outdated Show resolved Hide resolved
scripts/tests Outdated Show resolved Hide resolved
Copy link
Contributor

@hanno-becker hanno-becker left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd like to make the --cpu to tests bench optional and omit -march if -mcpu is specified.

@potsrevennil potsrevennil marked this pull request as draft June 20, 2024 06:08
@potsrevennil potsrevennil marked this pull request as ready for review June 20, 2024 09:32
Copy link
Contributor

@hanno-becker hanno-becker left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you @potsrevennil, I confirm that --arch-flags works.

May I ask for one more usability improvement: For perf, I have to run the binaries with sudo, but somehow sudo ./scripts/test ... doesn't work (and is also not ideal, because the build itself really needs not sudo rights).

Can you add an option to the test script which builds only, and prints the names of the files that have been produced?

@potsrevennil
Copy link
Contributor Author

@hanno-becker Yeah, I agree with you that it indeed makes more sense in some cases to separate building and running the binary. There're also cases that we would like to run the binaries in a different machine.

However, I'm thinking this might a bit out of scope for this PR, maybe we could open an issue to address this more, and make it a separate PR. What do you think ?

@hanno-becker
Copy link
Contributor

@potsrevennil Ok!

@hanno-becker hanno-becker merged commit a42e319 into pq-code-package:main Jun 20, 2024
7 checks passed
@potsrevennil potsrevennil deleted the bench-ci branch June 20, 2024 15:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
benchmark this PR should be benchmarked in CI
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants