Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

update: switch assets.yml to use GitHub API for artifact upload #4405

Merged
merged 1 commit into from
Oct 29, 2024

Conversation

yash-zededa
Copy link
Collaborator

Previous action for uploading artifacts to GitHub Releases failed due to parallel (matrix) execution issues. Replaced it with direct GitHub API calls to handle uploads.

This change provides more flexibility and control over artifact management and release handling.

@yash-zededa yash-zededa marked this pull request as ready for review October 28, 2024 17:22
@github-actions github-actions bot requested a review from eriknordmark October 28, 2024 17:50
RELEASE_ID: ${{ needs.create_release.outputs.release_id }}
UPLOAD_URL: ${{ needs.create_release.outputs.upload_url }}
run: |
for file in assets/*${{env.ARCH}}*; do
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since you already converted all of the assets to ARCH.*, can you not just do for file in assets/*?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Come to think of it, why even bother with the step "Rename files for release" and then this step? Just combine them:

for file in assets/*; do
    file_name=${{ env.ARCH }}.$(basename "$file")
    echo "Uploading $file as $file_name..."
    # do upload
done

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good suggestion! We can combine them, but since assets/${{ env.ARCH }}.rootfs.img.sha256 isn't part of the assets, I’ll just need to generate the sha256sum and make the necessary modifications.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@deitch combined the two loops and added a condition for the sha256 checksum for rootfs.img

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do the parallel runs in the matrix share their assets? If so the loop might pick up the assets from another run which isn't done yet.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nope, the parallel runs don't share the assets.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok

@yash-zededa yash-zededa force-pushed the assets-to-use-github-api branch from 775ee6b to 14d9b48 Compare October 28, 2024 17:56
@github-actions github-actions bot requested a review from deitch October 28, 2024 17:56
@yash-zededa yash-zededa force-pushed the assets-to-use-github-api branch 3 times, most recently from 3754c64 to 6033980 Compare October 28, 2024 18:42
run: |
# Create SHA256 checksum for rootfs.img
sha256sum "assets/rootfs.img" | awk '{ print $1 }' > "assets/${ARCH}.rootfs.img.sha256"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there any reason to make this the only file that already has the ${ARCH} at the beginning, and then you need the extra logic below? Can you do it more simply?

          sha256sum "assets/rootfs.img" | awk '{ print $1 }' > "assets/rootfs.img.sha256"
          for asset in assets/*; do
              base_name=$(basename "$asset")
              new_name="${ARCH}.${base_name}"  # Add ARCH prefix
              echo "Uploading $asset as $new_name..."
            upload_response=$(curl -s -X POST \
              -H "Authorization: Bearer $GITHUB_TOKEN" \
              -H "Content-Type: application/octet-stream" \
              --data-binary @"$asset" \
              "$UPLOAD_URL?name=$new_name")
    done

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

From what I understand, the assets/${ARCH}.rootfs.img.sha256 file is generated by the GitHub Actions workflow. If I create this file inside the loop, the loop may miss it and it won’t be uploaded. Therefore, I need to generate the checksum file first and include an if condition in the loop to ensure it doesn’t get renamed again.

I hope my understanding is correct and that I'm not going into loops.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You aren't doing it inside the loop, but just before the loop. This is all part of a single script, so you should be fine.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, I missed the last part. Yes, we can simplify this by creating a new file and letting the loop handle it.

I'm not sure why I didn't consider that and why I'm making things more complicated than necessary :)

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure why I didn't consider that and why I'm making things more complicated than necessary :)

human + engineer = make things complicated

just the way we are wired. 😄

@github-actions github-actions bot requested a review from deitch October 28, 2024 19:01
@yash-zededa yash-zededa force-pushed the assets-to-use-github-api branch from b8337e7 to ed4f9ab Compare October 28, 2024 19:02
Copy link
Contributor

@eriknordmark eriknordmark left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Once the review is done, can you make this PR be on top of 13.6? That way we can create a 13.6.1 and later delete it (if everything works) and put this in master and in 13.4-stable.

@github-actions github-actions bot requested a review from eriknordmark October 29, 2024 07:41
Previous action for uploading artifacts to GitHub Releases failed due to
parallel (matrix) execution issues. Replaced it with direct GitHub API
calls to handle uploads.

This change provides more flexibility and control over artifact
management and release handling.

Added step to generate sha256 checksum for rootfs.img

Signed-off-by: yash-zededa <[email protected]>
@yash-zededa yash-zededa force-pushed the assets-to-use-github-api branch from 9b5cb7f to e50388c Compare October 29, 2024 07:45
@yash-zededa yash-zededa changed the base branch from master to 13.6 October 29, 2024 09:36
@yash-zededa yash-zededa merged commit e875d32 into lf-edge:13.6 Oct 29, 2024
18 checks passed
@eriknordmark eriknordmark added the stable Should be backported to stable release(s) label Oct 29, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
stable Should be backported to stable release(s)
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants