Skip to content

Commit

Permalink
chore: Integrate benchmark output in GHA
Browse files Browse the repository at this point in the history
The code comes from ivanvc in etcd-io/bbolt#750
  • Loading branch information
mrueg committed Nov 6, 2024
1 parent 02b6a21 commit 0303532
Show file tree
Hide file tree
Showing 2 changed files with 17 additions and 3 deletions.
12 changes: 11 additions & 1 deletion .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -164,7 +164,17 @@ jobs:
- name: Benchmark tests
run: |
make test-benchmark-compare
BENCHSTAT_OUTPUT_FILE=result.txt make test-benchmark-compare
- run: |
echo "\`\`\`" >> "$GITHUB_STEP_SUMMARY"
cat result.txt >> "$GITHUB_STEP_SUMMARY"
echo "\`\`\`" >> "$GITHUB_STEP_SUMMARY"
cat <<EOL >> "$GITHUB_STEP_SUMMARY"
<hr />
The table shows the median and 95% confidence interval (CI) summaries for each benchmark comparing the HEAD and the BASE, and an A/B comparison under "vs base". The last column shows the statistical p-value with ten runs (n=10).
The last row has the Geometric Mean (geomean) for the given rows in the table.
Refer to [benchstat's documentation](https://pkg.go.dev/golang.org/x/perf/cmd/benchstat) for more help.
EOL
ci-build-kube-state-metrics:
name: ci-build-kube-state-metrics
Expand Down
8 changes: 6 additions & 2 deletions tests/compare_benchmarks.sh
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ set -u
[[ "$#" -le 2 ]] || echo "At least one argument required, $# provided."

REF_CURRENT="$(git rev-parse --abbrev-ref HEAD)"
REF_TO_COMPARE=$1
REF_TO_COMPARE="${1}"

COUNT=${2:-"1"}

Expand Down Expand Up @@ -42,4 +42,8 @@ echo ""
echo "### Result"
echo "old=${REF_TO_COMPARE} new=${REF_CURRENT}"

benchstat "${RESULT_TO_COMPARE}" "${RESULT_CURRENT}"
if [[ -z "${BENCHSTAT_OUTPUT_FILE}" ]]; then
benchstat "${REF_TO_COMPARE}=${RESULT_TO_COMPARE}" "${REF_CURRENT}=${RESULT_CURRENT}"
else
benchstat "${REF_TO_COMPARE}=${RESULT_TO_COMPARE}" "${REF_CURRENT}=${RESULT_CURRENT}" | tee -a "${BENCHSTAT_OUTPUT_FILE}"
fi

0 comments on commit 0303532

Please sign in to comment.