Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add codebuild integrations for our workflows #1254

Closed

Conversation

coderbirju
Copy link
Contributor

@coderbirju coderbirju commented May 29, 2024

Requires #1255
This PR adds the capability to run our workloads on codebuild instances. This allows us to manage the instance size/performance in a consistent manner. It also allows the use of ARM runners which will enable us to automate the release process for arm binaries in the future.

Running the Integration tests on both GH runners and codebuild to make sure we are able to identify any potential kernel specific issues.

Don't see any compelling reasons to run the integration tests on both github and codebuild and have kept the fork runs exclusively on github. If there are any kernel issues then we will see it when we make updates and run these tests in the fork.

We now use Ubuntu 22.04 as the underlying OS for all our workloads

Issue #, if available:

Description of changes:

Testing performed:

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

Signed-off-by: Arjun Raja Yogidas <[email protected]>
@coderbirju coderbirju force-pushed the codebuild-test-integration-failures branch from d7852e8 to 254485a Compare June 3, 2024 16:56
@coderbirju coderbirju marked this pull request as ready for review June 3, 2024 18:00
@coderbirju coderbirju requested a review from a team as a code owner June 3, 2024 18:00
Copy link
Contributor

@sondavidb sondavidb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IMO the default should be to run on GH Actions runners, and only run tests on CodeBuild if we have a reason.

With that reasoning, I think we should only run benchmarking (for consistency) + integration tests (for ARM support) on CodeBuild. Perhaps unit tests as well, just so we're running all of our SOCI-related tests in a consistent environment. But that's just my thoughts, feel free to disagree.

.github/workflows/releases.yml Outdated Show resolved Hide resolved
.github/workflows/benchmark_visualization.yml Outdated Show resolved Hide resolved
.github/workflows/build.yml Outdated Show resolved Hide resolved
.github/workflows/bump-deps.yml Outdated Show resolved Hide resolved
.github/workflows/prebuild.yml Outdated Show resolved Hide resolved
.github/workflows/releases.yml Show resolved Hide resolved
@austinvazquez austinvazquez added the github_actions Pull requests that update GitHub Actions code label Jun 4, 2024
@coderbirju coderbirju force-pushed the codebuild-test-integration-failures branch 6 times, most recently from ffa13d2 to 3eb8c7e Compare June 4, 2024 21:08
@@ -17,7 +17,7 @@ env:

jobs:
benchmark:
runs-on: ubuntu-20.04
runs-on: ${{ 'github.event.pull_request.head.repo.fork ' && 'ubuntu-22.04' || 'codebuild-soci-visualization-workflow-${{ github.run_id }}-${{ github.run_attempt }}-ubuntu-7.0-xlarge' }}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Based on the tests run in this PR it seems this will just always run 22.04.

Also, I assume we're switching to 22.04 because that's what CodeBuild uses?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated the workflow, there is not really a simpler way to add conditions to the runner type. One of those times where WET code makes more sense to reduce complicated conditions.

@coderbirju coderbirju force-pushed the codebuild-test-integration-failures branch 8 times, most recently from 2afb605 to 10e26ba Compare June 5, 2024 21:55
@coderbirju coderbirju force-pushed the codebuild-test-integration-failures branch from 10e26ba to 5032fbc Compare June 11, 2024 18:29
Kern--
Kern-- previously approved these changes Jun 11, 2024
sondavidb
sondavidb previously approved these changes Jun 11, 2024
@coderbirju coderbirju dismissed stale reviews from sondavidb and Kern-- via 9fe26cc June 20, 2024 22:43
@coderbirju coderbirju force-pushed the codebuild-test-integration-failures branch from 9fe26cc to 459324d Compare July 1, 2024 19:15
@coderbirju coderbirju force-pushed the codebuild-test-integration-failures branch from 459324d to 3ba1ee0 Compare July 1, 2024 20:06
@coderbirju coderbirju closed this Jul 15, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
github_actions Pull requests that update GitHub Actions code
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants