From 9126b9d3cdb83252c19cf5e7bc8b68b996758d90 Mon Sep 17 00:00:00 2001
From: "Thing-han, Lim" <15379156+potsrevennil@users.noreply.github.com>
Date: Mon, 18 Nov 2024 16:10:26 +0800
Subject: [PATCH] ci: Add benchmark regression alert

The threshold is set to 103%, which means that if the previous benchmark
result is 100 cycles, then there should be an GH comment alert in the PR if
the benchmark result is > 103 cycles.

For the `comment-on-alert` to work on PR, we cannot skip the `store
benchmark result` step.
We could avoid pushing to `gh-pages` branch directly from PR by configuring
`auto-push` to `false` if not on `main` branch.

Signed-off-by: Thing-han, Lim <15379156+potsrevennil@users.noreply.github.com>
---
 .github/actions/bench/action.yml | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)

diff --git a/.github/actions/bench/action.yml b/.github/actions/bench/action.yml
index c7f38eb9b..5ed1a2e2f 100644
--- a/.github/actions/bench/action.yml
+++ b/.github/actions/bench/action.yml
@@ -84,11 +84,13 @@ runs:
       run: |
         check-namespace
     - name: Store benchmark result
-      if: ${{ inputs.store_results == 'true' }}
       uses: benchmark-action/github-action-benchmark@v1
       with:
         name: ${{ inputs.name }}
         tool: "customSmallerIsBetter"
         output-file-path: output.json
         github-token: ${{ inputs.gh_token }}
-        auto-push: true
+        auto-push: ${{ inputs.store_results == 'true' }}
+        comment-on-alert: true
+        summary-always: true
+        alert-threshold: 103%