Skip to content
This repository has been archived by the owner on Oct 11, 2024. It is now read-only.

Commit

Permalink
Add timeouts to CI steps (#4563)
Browse files Browse the repository at this point in the history
This should help us catch any CI steps that stall. The timeouts should set should be more than enough for most cases and is more than 4x the average time it takes.

https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#jobsjob_idstepstimeout-minutes

Quick tasks: 10m
Medium tasks (eg: linting website, code): 30m
Integration tests: 120m

<!-- PR description-->

---

#### Does this PR need a docs update or release note?

- [ ] ✅ Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x] ⛔ No

#### Type of change

<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [x] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

#### Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* #<issue>

#### Test Plan

<!-- How will this be tested prior to merging.-->
- [x] 💪 Manual
- [ ] ⚡ Unit test
- [ ] 💚 E2E
  • Loading branch information
meain authored Oct 27, 2023
1 parent be07a04 commit fd887f4
Show file tree
Hide file tree
Showing 3 changed files with 46 additions and 0 deletions.
5 changes: 5 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -110,6 +110,7 @@ jobs:
needs: [Precheck, Checkout, SetEnv]
environment: Testing
runs-on: ubuntu-latest
timeout-minutes: 30
if: startsWith(github.ref, 'refs/tags/') || github.ref == 'refs/heads/main' || needs.precheck.outputs.websitefileschanged == 'true' # websitefileschanged also includes srcfileschanged

steps:
Expand All @@ -128,6 +129,7 @@ jobs:
needs: [Precheck, Checkout, SetM365App]
environment: Testing
runs-on: ubuntu-latest
timeout-minutes: 120
if: (startsWith(github.ref, 'refs/tags/') || github.ref == 'refs/heads/main') || (needs.precheck.outputs.srcfileschanged == 'true' && github.event.pull_request.head.repo.full_name == github.repository)
defaults:
run:
Expand Down Expand Up @@ -198,6 +200,7 @@ jobs:
needs: [Precheck, Checkout, SetM365App]
environment: Testing
runs-on: ubuntu-latest
timeout-minutes: 30
if: (startsWith(github.ref, 'refs/tags/') || github.ref == 'refs/heads/main') || (needs.precheck.outputs.srcfileschanged == 'true' && github.event.pull_request.head.repo.full_name == github.repository)
defaults:
run:
Expand Down Expand Up @@ -268,6 +271,7 @@ jobs:
needs: [Precheck, Checkout]
environment: Testing
runs-on: ubuntu-latest
timeout-minutes: 30
if: needs.precheck.outputs.srcfileschanged == 'true'
defaults:
run:
Expand Down Expand Up @@ -445,6 +449,7 @@ jobs:
needs: [Precheck, Checkout]
environment: Testing
runs-on: ubuntu-latest
timeout-minutes: 30
if: startsWith(github.ref, 'refs/tags/') || github.ref == 'refs/heads/main' || needs.precheck.outputs.srcfileschanged == 'true'
defaults:
run:
Expand Down
15 changes: 15 additions & 0 deletions .github/workflows/longevity_test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -77,13 +77,15 @@ jobs:
git checkout ${{ github.ref }} -- .github
- run: go build -o corso
timeout-minutes: 10

- run: mkdir ${CORSO_LOG_DIR}

# Use shorter-lived credentials obtained from assume-role since these
# runs haven't been taking long.
- name: Configure AWS credentials from Test account
uses: aws-actions/configure-aws-credentials@v4
timeout-minutes: 10
with:
role-to-assume: ${{ secrets.AWS_IAM_ROLE }}
role-session-name: integration-testing
Expand All @@ -93,11 +95,13 @@ jobs:
# Repository commands

- name: Version Test
timeout-minutes: 10
run: |
./corso --version | grep -c 'Corso version:'
- name: Repo init test
id: repo-init
timeout-minutes: 10
run: |
set -euo pipefail
echo -e "\nRepo init test\n" >> ${{ env.CORSO_LOG_FILE }}
Expand All @@ -119,6 +123,7 @@ jobs:
fi
- name: Repo connect test
timeout-minutes: 10
run: |
set -euo pipefail
echo -e "\nRepo connect test\n" >> ${{ env.CORSO_LOG_FILE }}
Expand All @@ -140,6 +145,7 @@ jobs:

- name: Backup exchange test
id: exchange-test
timeout-minutes: 30
run: |
echo -e "\nBackup Exchange test\n" >> ${CORSO_LOG_FILE}
./corso backup create exchange \
Expand All @@ -164,6 +170,7 @@ jobs:

- name: Backup onedrive test
id: onedrive-test
timeout-minutes: 30
run: |
set -euo pipefail
echo -e "\nBackup OneDrive test\n" >> ${CORSO_LOG_FILE}
Expand All @@ -188,6 +195,7 @@ jobs:
# Sharepoint test
- name: Backup sharepoint test
id: sharepoint-test
timeout-minutes: 30
run: |
set -euo pipefail
echo -e "\nBackup SharePoint test\n" >> ${CORSO_LOG_FILE}
Expand All @@ -213,6 +221,7 @@ jobs:
# Backup Exchange Deletion test
- name: Backup Delete exchange test
id: delete-exchange-test
timeout-minutes: 30
env:
SERVICE: "exchange"
DELETION_DAYS: 10
Expand All @@ -225,6 +234,7 @@ jobs:
# Backup Onedrive Deletion test
- name: Backup Delete onedrive test
id: delete-onedrive-test
timeout-minutes: 30
env:
SERVICE: "onedrive"
DELETION_DAYS: 10
Expand All @@ -237,6 +247,7 @@ jobs:
# Backup Sharepoint Deletion test
- name: Backup Delete Sharepoint test
id: delete-sharepoint-test
timeout-minutes: 30
env:
SERVICE: "sharepoint"
DELETION_DAYS: 5
Expand All @@ -248,6 +259,7 @@ jobs:
##########################################################################
# Export OneDrive Test
- name: OneDrive Export test
timeout-minutes: 30
run: |
set -euo pipefail
echo -e "\Export OneDrive test\n" >> ${CORSO_LOG_FILE}
Expand Down Expand Up @@ -275,6 +287,7 @@ jobs:
##########################################################################
# Export SharePoint Test
- name: SharePoint Export test
timeout-minutes: 30
run: |
set -euo pipefail
echo -e "\Export SharePoint test\n" >> ${CORSO_LOG_FILE}
Expand Down Expand Up @@ -303,6 +316,7 @@ jobs:
# Maintenance test
- name: Maintenance test Daily
id: maintenance-test-daily
timeout-minutes: 30
run: |
set -euo pipefail
echo -e "\n Maintenance test Daily\n" >> ${CORSO_LOG_FILE}
Expand All @@ -319,6 +333,7 @@ jobs:
- name: Maintenance test Weekly
id: maintenance-test-weekly
timeout-minutes: 30
run: |
if [[ $(date +%A) == "Saturday" ]]; then
set -euo pipefail
Expand Down
26 changes: 26 additions & 0 deletions .github/workflows/sanity-test.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -57,7 +57,10 @@ jobs:
go-version-file: src/go.mod

- run: go build -o corso
timeout-minutes: 10

- run: go build -o sanity-test ./cmd/sanity_test
timeout-minutes: 10

- run: mkdir ${CORSO_LOG_DIR}

Expand All @@ -77,6 +80,7 @@ jobs:
echo "NOW=$(date +"%Y-%m-%dT%H:%M:%SZ")" >> $GITHUB_ENV
- name: Purge CI-Produced Folders for Users
timeout-minutes: 30
uses: ./.github/actions/purge-m365-data
with:
user: ${{ env.TEST_USER }}
Expand All @@ -89,6 +93,7 @@ jobs:
m365-admin-password: ${{ secrets.M365_TENANT_ADMIN_PASSWORD }}

- name: Purge CI-Produced Folders for Sites
timeout-minutes: 30
if: always()
uses: ./.github/actions/purge-m365-data
with:
Expand All @@ -107,10 +112,12 @@ jobs:
# Repository commands

- name: Version Test
timeout-minutes: 10
run: |
./corso --version | grep -c 'Corso version:'
- name: Repo init test
timeout-minutes: 10
id: repo-init
run: |
set -euo pipefail
Expand All @@ -132,6 +139,7 @@ jobs:
echo result="$prefix" >> $GITHUB_OUTPUT
- name: Repo connect test
timeout-minutes: 10
run: |
set -euo pipefail
echo -e "\nRepo connect test\n" >> ${{ env.CORSO_LOG_FILE }}
Expand All @@ -151,6 +159,7 @@ jobs:
# Run maintenance on an empty repo just to make sure the command still
# works.
- name: Repo maintenance test
timeout-minutes: 30
run: |
set -euo pipefail
echo -e "\nRepo maintenance test\n" >> ${{ env.CORSO_LOG_FILE }}
Expand All @@ -167,6 +176,7 @@ jobs:
# generate new entries to roll into the next load test
# only runs if the test was successful
- name: Exchange - Create new data
timeout-minutes: 30
working-directory: ./src/cmd/factory
run: |
go run . exchange emails \
Expand All @@ -176,6 +186,7 @@ jobs:
--count 4
- name: Exchange - Backup
timeout-minutes: 30
id: exchange-backup
uses: ./.github/actions/backup-restore-test
with:
Expand All @@ -187,6 +198,7 @@ jobs:
log-dir: ${{ env.CORSO_LOG_DIR }}

- name: Exchange - Incremental backup
timeout-minutes: 30
id: exchange-backup-incremental
uses: ./.github/actions/backup-restore-test
with:
Expand All @@ -199,6 +211,7 @@ jobs:
log-dir: ${{ env.CORSO_LOG_DIR }}

- name: Exchange - Non delta backup
timeout-minutes: 30
id: exchange-backup-non-delta
uses: ./.github/actions/backup-restore-test
with:
Expand All @@ -211,6 +224,7 @@ jobs:
log-dir: ${{ env.CORSO_LOG_DIR }}

- name: Exchange - Incremental backup after non-delta
timeout-minutes: 30
id: exchange-backup-incremental-after-non-delta
uses: ./.github/actions/backup-restore-test
with:
Expand All @@ -230,6 +244,7 @@ jobs:
# generate new entries for test
- name: OneDrive - Create new data
id: new-data-creation-onedrive
timeout-minutes: 30
working-directory: ./src/cmd/factory
run: |
suffix=$(date +"%Y-%m-%d_%H-%M-%S")
Expand All @@ -245,6 +260,7 @@ jobs:
- name: OneDrive - Backup
id: onedrive-backup
timeout-minutes: 30
uses: ./.github/actions/backup-restore-test
with:
service: onedrive
Expand All @@ -257,6 +273,7 @@ jobs:

# generate some more enteries for incremental check
- name: OneDrive - Create new data (for incremental)
timeout-minutes: 30
working-directory: ./src/cmd/factory
run: |
go run . onedrive files \
Expand All @@ -268,6 +285,7 @@ jobs:
- name: OneDrive - Incremental backup
id: onedrive-incremental
timeout-minutes: 30
uses: ./.github/actions/backup-restore-test
with:
service: onedrive
Expand All @@ -285,6 +303,7 @@ jobs:
# generate new entries for test
- name: SharePoint - Create new data
id: new-data-creation-sharepoint
timeout-minutes: 30
working-directory: ./src/cmd/factory
run: |
suffix=$(date +"%Y-%m-%d_%H-%M-%S")
Expand All @@ -301,6 +320,7 @@ jobs:
- name: SharePoint - Backup
id: sharepoint-backup
timeout-minutes: 30
uses: ./.github/actions/backup-restore-test
with:
service: sharepoint
Expand All @@ -313,6 +333,7 @@ jobs:

# generate some more enteries for incremental check
- name: SharePoint - Create new data (for incremental)
timeout-minutes: 30
working-directory: ./src/cmd/factory
run: |
go run . sharepoint files \
Expand All @@ -325,6 +346,7 @@ jobs:
- name: SharePoint - Incremental backup
id: sharepoint-incremental
timeout-minutes: 30
uses: ./.github/actions/backup-restore-test
with:
service: sharepoint
Expand All @@ -342,6 +364,7 @@ jobs:
# generate new entries for test
- name: Groups - Create new data
id: new-data-creation-groups
timeout-minutes: 30
working-directory: ./src/cmd/factory
run: |
suffix=$(date +"%Y-%m-%d_%H-%M-%S")
Expand All @@ -358,6 +381,7 @@ jobs:
- name: Groups - Backup
id: groups-backup
timeout-minutes: 30
uses: ./.github/actions/backup-restore-test
with:
service: groups
Expand All @@ -369,6 +393,7 @@ jobs:

# generate some more enteries for incremental check
- name: Groups - Create new data (for incremental)
timeout-minutes: 30
working-directory: ./src/cmd/factory
run: |
go run . sharepoint files \
Expand All @@ -381,6 +406,7 @@ jobs:
- name: Groups - Incremental backup
id: groups-incremental
timeout-minutes: 30
uses: ./.github/actions/backup-restore-test
with:
service: groups
Expand Down

0 comments on commit fd887f4

Please sign in to comment.