ci: fix CI reports not uploading #138

aaronsteers · 2024-12-05T19:43:25Z

This resolves the issue where airbyte-ci reports were not being uploaded, and the PR author would have to manually download the job results artifact, then decompress, and then navigate to the report url.

Requires this PR to be merged:

airbyte-ci: don't require extra permissions or --is-ci flag to upload reports to GCS bucket airbyte#48824

Now, the report URL is printed in the job execution log (as usual for Airbyte CI tests):

For convenient access, it is also posted to the "Summary" page of the GitHub Workflow for easy access, and to prevent having to scan through job logs:

https://github.com/airbytehq/airbyte-python-cdk/actions/runs/12187332389?pr=138

The job artifacts are still downloadable from the bottom of the Summary page as well, in case other test report artifacts are useful - but for most cases, you can simply go to the summary page and click on the report link in the job summary.

Summary by CodeRabbit

New Features
- Enhanced reporting capabilities with HTML report URL extraction.
- Improved clarity in workflow execution outputs.
Bug Fixes
- Clearer output messages when the job is aborted due to no relevant changes.
Chores
- Updated job permissions to allow creating checks on pull requests.
- Specified branch for code checkout in testing.
- Adjusted job name for conditional skipping based on relevant changes.

aaronsteers · 2024-12-05T20:26:40Z

/autofix

Auto-Fix Job Info

This job attempts to auto-fix any linting or formating issues. If any fixes are made,
those changes will be automatically committed and pushed back to the PR.

Note: This job can only be run by maintainers. On PRs from forks, this command requires
that the PR author has enabled the Allow edits from maintainers option.

PR auto-fix job started... Check job output.

✅ Changes applied successfully.

.github/workflows/connector-tests.yml

coderabbitai · 2024-12-05T20:30:10Z

📝 Walkthrough

Walkthrough

The pull request modifies the .github/workflows/connector-tests.yml workflow file to enhance its functionality. Key changes include updates to job permissions, conditional job naming based on previous job outputs, and improved output messaging for clarity. The checkout step now references a specific branch, and the Test Connector step has been adjusted with additional parameters. The output handling has been refined to include an HTML report URL and success status, improving the workflow's reporting capabilities.

Changes

File	Change Summary
.github/workflows/connector-tests.yml	- Updated `permissions` to include `checks: write` for `connectors_ci` job.
	- Modified job name to include conditional skip message based on `cdk_changes` job outputs.
	- Refined `Abort if extra not changed` step for clearer output messages.
	- Changed checkout step to reference branch `aj/airbyte-ci/allow-report-uploads-with-fewer-permissions`.
	- Adjusted `Test Connector` step to include `--ci-report-bucket-name=airbyte-ci-reports-multi`.
	- Renamed `Evaluate Test Output` to `Evaluate Test` and added `id: evaluate_output`.
	- Added output variables `success` and `html_report_url` in `Evaluate Test` step.
	- Enhanced job output to include success status and HTML report URL for further use.

Possibly related PRs

Feat: Incorporate SDM in CDK and add publish workflow #58: Enhances the connectors_ci job in the .github/workflows/connector-tests.yml file, directly related to the main PR's job conditions and output handling.
ci: run airbyte connector tests from master branch #99: Updates the connectors_ci job with a focus on job cancellation logic based on source code changes, aligning with the main PR's changes.
ci: skip downstream connector tests if no source code is changed #101: Refines job conditions for the connectors_ci job, similar to the enhancements made in the main PR regarding job abort logic.
ci: reorder job steps #102: Reorders steps within the connectors_ci job, relating to structural changes made in the main PR to improve clarity and functionality.

Suggested reviewers

aaronsteers

📜 Recent review details

Configuration used: CodeRabbit UI
Review profile: CHILL

📥 Commits

Reviewing files that changed from the base of the PR and between 491896d and 9748eae.

📒 Files selected for processing (1)

.github/workflows/connector-tests.yml (4 hunks)

🚧 Files skipped from review as they are similar to previous changes (1)

.github/workflows/connector-tests.yml

Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media?

❤️ Share

🪧 Tips

Chat

There are 3 ways to chat with CodeRabbit:

‼️ IMPORTANT
Auto-reply has been disabled for this repository in the CodeRabbit settings. The CodeRabbit bot will not respond to your replies unless it is explicitly tagged.

Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
- @coderabbitai generate unit testing code for this file.
- @coderabbitai modularize this function.
PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
- @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
- @coderabbitai read src/utils.ts and generate unit testing code.
- @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
- @coderabbitai help me debug CodeRabbit configuration file.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (Invoked using PR comments)

@coderabbitai pause to pause the reviews on a PR.
@coderabbitai resume to resume the paused reviews.
@coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
@coderabbitai full review to do a full review from scratch and review all the files again.
@coderabbitai summary to regenerate the summary of the PR.
@coderabbitai resolve resolve all the CodeRabbit review comments.
@coderabbitai configuration to show the current CodeRabbit configuration for the repository.
@coderabbitai help to get help.

Other keywords and placeholders

Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.
Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
Add @coderabbitai anywhere in the PR title to generate the title automatically.

CodeRabbit Configuration File (`.coderabbit.yaml`)

You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
Please see the configuration documentation for more information.
If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json

Documentation and Community

Visit our Documentation for detailed information on how to use CodeRabbit.
Join our Discord Community to get help, request features, and share feedback.
Follow us on X/Twitter for updates and announcements.

coderabbitai

Actionable comments posted: 1

🧹 Outside diff range and nitpick comments (1)

.github/workflows/connector-tests.yml (1)
136-138: Consider parameterizing the bucket name? 🤔

The hardcoded bucket name works, but would it be more flexible to move this to a GitHub secret or environment variable? This would make it easier to use different buckets for different environments, wdyt?
-          --ci-report-bucket-name=airbyte-ci-reports-multi \
+          --ci-report-bucket-name=${{ secrets.CI_REPORT_BUCKET_NAME }} \

📜 Review details

Configuration used: CodeRabbit UI
Review profile: CHILL

📥 Commits

Reviewing files that changed from the base of the PR and between 819555f and 491896d.

📒 Files selected for processing (1)

.github/workflows/connector-tests.yml (5 hunks)

🔇 Additional comments (2)

.github/workflows/connector-tests.yml (2)

97-98: LGTM! The checks permission is necessary.

The addition of checks: write permission is appropriate for creating check runs and aligns with the goal of improving report accessibility.

Line range hint 146-170: Great job on the enhanced reporting! 👏

The changes nicely improve the visibility of test results by:

Extracting and displaying the HTML report URL
Adding clear success/failure indicators
Providing structured outputs for downstream steps

This aligns perfectly with the PR's goal of making reports more accessible.

.github/workflows/connector-tests.yml

natikgadzhi

Can we also make the GH check line link to the html report?

Approving contingent on the comment that you made to revert the thing to master

aaronsteers · 2024-12-05T21:13:44Z

Can we also make the GH check line link to the html report?

@natikgadzhi - Yes, I think so. I wanted to implement the same but my attempt was reverted here (a50c1b7) due to a "not found" error when I tested it. I'll create a follow-on issue to try again in another PR...

Update: logged here:

CI: Try to replace "details" link on "Connector Test" check results to go directly to the HTML report output #139

ci: provide a gcs report bucket name

d0f4120

github-actions bot added the ci label Dec 5, 2024

aaronsteers added 3 commits December 5, 2024 11:44

dummy change

57d4683

fix airbyte-ci syntax

3899e68

evaluate results and post the url for easy viewing

44a5a56

aaronsteers mentioned this pull request Dec 5, 2024

airbyte-ci: don't require extra permissions or --is-ci flag to upload reports to GCS bucket airbytehq/airbyte#48824

Merged

2 tasks

remove details url override attempt (not working)

a50c1b7

aaronsteers requested review from bnchrch, brianjlai, natikgadzhi and alafanechere December 5, 2024 20:24

aaronsteers marked this pull request as ready for review December 5, 2024 20:25

Auto-fix lint and format issues

491896d

aaronsteers commented Dec 5, 2024

View reviewed changes

.github/workflows/connector-tests.yml Outdated Show resolved Hide resolved

aaronsteers commented Dec 5, 2024

View reviewed changes

.github/workflows/connector-tests.yml Show resolved Hide resolved

coderabbitai bot requested changes Dec 5, 2024

View reviewed changes

.github/workflows/connector-tests.yml Outdated Show resolved Hide resolved

natikgadzhi approved these changes Dec 5, 2024

View reviewed changes

Apply suggestions from code review

9748eae

aaronsteers mentioned this pull request Dec 5, 2024

CI: Try to replace "details" link on "Connector Test" check results to go directly to the HTML report output #139

Open

coderabbitai bot approved these changes Dec 5, 2024

View reviewed changes

aaronsteers merged commit ff62936 into main Dec 5, 2024
18 checks passed

aaronsteers deleted the aj/ci/fix-ci-reports-not-uploading branch December 5, 2024 21:18

aaronsteers mentioned this pull request Dec 5, 2024

Bring back html reports being uploaded to public bucket #137

Closed

This was referenced Dec 6, 2024

ci: update connector-tests.yml worfklow to use dev binary of airbyte-ci #131

Merged

ci: make it easier to find the connector test report url upon failure #153

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

ci: fix CI reports not uploading #138

ci: fix CI reports not uploading #138

aaronsteers commented Dec 5, 2024 •

edited by coderabbitai bot

Loading

aaronsteers commented Dec 5, 2024 •

edited by github-actions bot

Loading

coderabbitai bot commented Dec 5, 2024 •

edited

Loading

Walkthrough

Changes

Possibly related PRs

Suggested reviewers

Chat

CodeRabbit Commands (Invoked using PR comments)

Other keywords and placeholders

CodeRabbit Configuration File (`.coderabbit.yaml`)

Documentation and Community

coderabbitai bot left a comment

natikgadzhi left a comment

aaronsteers commented Dec 5, 2024 •

edited

Loading

ci: fix CI reports not uploading #138

ci: fix CI reports not uploading #138

Conversation

aaronsteers commented Dec 5, 2024 • edited by coderabbitai bot Loading

Summary by CodeRabbit

aaronsteers commented Dec 5, 2024 • edited by github-actions bot Loading

coderabbitai bot commented Dec 5, 2024 • edited Loading

Walkthrough

Changes

Possibly related PRs

Suggested reviewers

Chat

CodeRabbit Commands (Invoked using PR comments)

Other keywords and placeholders

CodeRabbit Configuration File (.coderabbit.yaml)

Documentation and Community

coderabbitai bot left a comment

Choose a reason for hiding this comment

natikgadzhi left a comment

Choose a reason for hiding this comment

aaronsteers commented Dec 5, 2024 • edited Loading

aaronsteers commented Dec 5, 2024 •

edited by coderabbitai bot

Loading

aaronsteers commented Dec 5, 2024 •

edited by github-actions bot

Loading

coderabbitai bot commented Dec 5, 2024 •

edited

Loading

CodeRabbit Configuration File (`.coderabbit.yaml`)

aaronsteers commented Dec 5, 2024 •

edited

Loading