ci: Add benchmark regression alert

The threshold is set to 103%, which means that if the previous benchmark result is 100 cycles, then there should be an GH comment alert in the PR if the benchmark result is > 103 cycles. For the `comment-on-alert` to work on PR, we cannot skip the `store benchmark result` step. We could avoid pushing to `gh-pages` branch directly from PR by configuring `auto-push` to `false` if not on `main` branch. Signed-off-by: Thing-han, Lim <[email protected]>
pq-code-package · Nov 19, 2024 · 9126b9d · 9126b9d
1 parent ea2193a
commit 9126b9d
Showing 1 changed file with 4 additions and 2 deletions.
diff --git a/.github/actions/bench/action.yml b/.github/actions/bench/action.yml
@@ -84,11 +84,13 @@ runs:
       run: |
         check-namespace
     - name: Store benchmark result
-      if: ${{ inputs.store_results == 'true' }}
       uses: benchmark-action/github-action-benchmark@v1
       with:
         name: ${{ inputs.name }}
         tool: "customSmallerIsBetter"
         output-file-path: output.json
         github-token: ${{ inputs.gh_token }}
-        auto-push: true
+        auto-push: ${{ inputs.store_results == 'true' }}
+        comment-on-alert: true
+        summary-always: true
+        alert-threshold: 103%