CI: Push non-opt benchmarks to GH pages #415
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
hanno-becker
force-pushed
the
triggerable_bench_action_fix
branch
from
fb88e02 to
f2cf9c8
Compare
hanno-becker
force-pushed
the
triggerable_bench_action_fix
branch
from
f2cf9c8 to
7622938
Compare
The manually triggered benchmarking workflow in `bench_ec2_any.yml` does not have sufficient permissions to call the underlying workhorse `bench_ec2_reusable.yml`. This commit fixes that. Signed-off-by: Hanno Becker <[email protected]>
Until now, we only pushed benchmarks for optimized code to the github pages. That was the wrong decision, as we are now curious to understand (a) where mlkem-native's pure-C performance stands in comparison to the reference implementation, and (b) how it has evolved over time, esp. in light of restructurings we made for the CBMC proofs. Ultimately, we should retroactively conduct & push non-opt benchmarks. For now, this commit merely enables pushing benchmarking results to GH pages for non-optimized code. Care needs to be taken to not mess up with existing optimized benchmarks by changing their name. Thus, we keep the previous names (which matched the target name, e.g. "Graviton 3"), but append " (no-opt)" in case we benchmarked the C backend. Signed-off-by: Hanno Becker <[email protected]>
hanno-becker
force-pushed
the
triggerable_bench_action_fix
branch
from
7622938 to
602ebfc
Compare
mkannwischer
approved these changes
Nov 17, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Until now, we only pushed benchmarks for optimized code to the github pages. That was the wrong decision, as we are now curious to understand (a) where mlkem-native's pure-C performance stands in comparison to the reference implementation, and (b) how it has evolved over time, esp. in light of restructurings we made for the CBMC proofs.
Ultimately, we should retroactively conduct & push non-opt benchmarks. For now, this commit merely enables pushing benchmarking results to GH pages for non-optimized code.
Care needs to be taken to not mess up with existing optimized benchmarks by changing their name. Thus, we keep the previous names (which matched the target name, e.g. "Graviton 3"), but append " (no-opt)" in case we benchmarked the C backend.