ci: add retry for pulling fluentd image from ECR Public (#76)

*Description of changes:* To fix the flaky scenario: https://github.com/aws/shim-loggers-for-containerd/actions/runs/6267460590/job/17020568425?pr=75 By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice. Signed-off-by: Ziwen Ning <[email protected]>
aws · Sep 25, 2023 · 183f239 · 183f239
1 parent 86ab73c
commit 183f239
Showing 1 changed file with 10 additions and 0 deletions.
diff --git a/.github/workflows/ci.yaml b/.github/workflows/ci.yaml
@@ -113,6 +113,16 @@ jobs:
           FLUENTD_LOG_DIR=${GITHUB_WORKSPACE}/fluentd-logs
           FLUENTD_PORT=24224
           FLUENTD_IMAGE=public.ecr.aws/docker/library/fluentd:v1.16-debian-1
+          : # ECR Public anonymous pull rate quota is 1 per sec. https://docs.aws.amazon.com/AmazonECR/latest/public/public-service-quotas.html.
+          : # Running multiple jobs at the same time may exceed rate limit. Retry 3 times to prevent flaky behavior.
+          count=0
+          until [ $count -ge 3 ]
+          do
+            docker pull $FLUENTD_IMAGE && break
+            count=$((count+1))
+            [ $count -eq 3 ] && echo "Pull command failed after 3 attempts" && exit 1
+            sleep 3
+          done
           : # Fluentd container is not using root user so need 777 to make it writable. https://docs.github.com/en/actions/using-github-hosted-runners/about-github-hosted-runners#docker-container-filesystem
           sudo mkdir -m 777 $FLUENTD_LOG_DIR
           docker run -d -p $FLUENTD_PORT:24224 -p $FLUENTD_PORT:24224/udp -v $FLUENTD_LOG_DIR:/fluentd/log $FLUENTD_IMAGE