Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

to trigger eden workflow based on condition #3450

Merged
merged 2 commits into from
Oct 9, 2023
Merged

to trigger eden workflow based on condition #3450

merged 2 commits into from
Oct 9, 2023

Conversation

yash-zededa
Copy link
Collaborator

@yash-zededa yash-zededa commented Sep 18, 2023

PR to use the new eden workflows for testing eve changes in PR, master and tag builds. Added concurrency across the workflow with cancel-in-progress: to avoid clogging of the builds. I have used multiple workflow triggers based on different conditions because github actions doesn't supports if else condition as per my understanding.

The run shell is an option but then we have to specificy the attribute runs-on which will cause delay in the trigger(based on the workload)

Please feel free to offer any other suitable options.

@codecov
Copy link

codecov bot commented Sep 18, 2023

Codecov Report

All modified lines are covered by tests ✅

Comparison is base (e1538b5) 20.54% compared to head (51f665b) 20.54%.

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #3450      +/-   ##
==========================================
- Coverage   20.54%   20.54%   -0.01%     
==========================================
  Files         202      202              
  Lines       45605    45605              
==========================================
- Hits         9371     9369       -2     
- Misses      35533    35536       +3     
+ Partials      701      700       -1     

see 1 file with indirect coverage changes

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Copy link
Member

@uncleDecart uncleDecart left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, couple comments:

  1. I'd switch workflows so that there will be no two testing workflows running in paralllel
  2. I'm not sure I understand how concurrency will work in case somebody will re-approve same PR I guess it'll re-trigger tests, which is not desirable, but tolerable for now. I did some digging and there's this fancy github.event.before and github.event.after which we can use to determine if git HEAD has changed, i.e.
      - name: Check if PR is Approved and No New Commits
        id: approval_and_no_new_commits
        run: |
          if [ "${{ github.event_name }}" == "pull_request" ] && [ "${{ github.event.action }}" != "closed" ]; then
            if [ "${{ github.event.pull_request.labels.*.name }}" == "approved" ] && [ "${{ github.event.before }}" == "${{ github.event.after }}" ]; then
              echo "PR is approved and has no new commits."
              echo "::set-output name=approved::true"
            else
              echo "PR is not approved or has new commits."
              echo "::set-output name=approved::false"
            fi
          fi
        shell: bash

      - name: Run Tests
        if: steps.approval_and_no_new_commits.outputs.approved == 'true'
        uses: lf-edge/eden/.github/workflows/test.yml@master
        with:
          eve_image: //...

But if I understand concurrency tag, if we add this and somebody will re-approve concurrency will stop previous job and new will not be triggered since there are no changes in PR. Alternative to it would be to remove concurrency group but then we will have blank jobs, when somebody re-approved job. What are your thoughts @yash-zededa ?

@yash-zededa
Copy link
Collaborator Author

LGTM, couple comments:

  1. I'd switch workflows so that there will be no two testing workflows running in paralllel
  2. I'm not sure I understand how concurrency will work in case somebody will re-approve same PR I guess it'll re-trigger tests, which is not desirable, but tolerable for now. I did some digging and there's this fancy github.event.before and github.event.after which we can use to determine if git HEAD has changed, i.e.
      - name: Check if PR is Approved and No New Commits
        id: approval_and_no_new_commits
        run: |
          if [ "${{ github.event_name }}" == "pull_request" ] && [ "${{ github.event.action }}" != "closed" ]; then
            if [ "${{ github.event.pull_request.labels.*.name }}" == "approved" ] && [ "${{ github.event.before }}" == "${{ github.event.after }}" ]; then
              echo "PR is approved and has no new commits."
              echo "::set-output name=approved::true"
            else
              echo "PR is not approved or has new commits."
              echo "::set-output name=approved::false"
            fi
          fi
        shell: bash

      - name: Run Tests
        if: steps.approval_and_no_new_commits.outputs.approved == 'true'
        uses: lf-edge/eden/.github/workflows/test.yml@master
        with:
          eve_image: //...

But if I understand concurrency tag, if we add this and somebody will re-approve concurrency will stop previous job and new will not be triggered since there are no changes in PR. Alternative to it would be to remove concurrency group but then we will have blank jobs, when somebody re-approved job. What are your thoughts @yash-zededa ?

@uncleDecart I think the issue only would be with the PR builds, we can have few more checks but that would involve use of a runner and that in my opinion would contribute to a delay. I think using labels would be an ideal approach, the tests would be triggered on PR only when there is a specific label is present. master branch and tag build should not cause any issue in my opinion. Moving forward, we can also label the PR automatically based on certain conditions - this will also reduce the congestion due to PR builds.

@uncleDecart
Copy link
Member

Update: did experiment with new workflows: created PR in fork with changes and approved it twice and it triggered tests twice (1 and 2), which is not desirable.

Will test labels now, if they will work then in order for tests to be run we will have to put labels manually, then we can automatize this by adding github bot.

Copy link
Contributor

@eriknordmark eriknordmark left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM (assuming we do the label stuff in a separate PR)

@yash-zededa
Copy link
Collaborator Author

yash-zededa commented Sep 20, 2023

Update:

The label trigger worked as expected but the combination of approved + labelled is not working. The pull request approval event should have the label info in it for the PR to be build - which is not desriable.

I am testing the https://github.com/abinoda/label-when-approved-action - if this works then we can test the same.

Copy link
Contributor

@eriknordmark eriknordmark left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note that DCO is failing since none of the commits are signed with your name and email. All of the commits need to be signed. git commit -S does that.

@yash-zededa
Copy link
Collaborator Author

Note that DCO is failing since none of the commits are signed with your name and email. All of the commits need to be signed. git commit -S does that.

Thanks, at times I forget to use -s. Signed all the commits. I will squash all the commits once testing is completed.

@yash-zededa
Copy link
Collaborator Author

yash-zededa commented Sep 26, 2023

Update:

We attempted to use https://github.com/marketplace/actions/label-approved-pull-requests, but it did not work properly. Although the PR was marked test-eden, the github actions were not triggered.

As of now, we will proceed with the understanding that re-approval of the same PR will result in a new process run and the previous will be cancelled.

Copy link
Contributor

@eriknordmark eriknordmark left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Copy link
Member

@uncleDecart uncleDecart left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@eriknordmark
Copy link
Contributor

We attempted to use https://github.com/marketplace/actions/label-approved-pull-requests, but it did not work properly. Although the PR was marked test-eden, the github actions were not triggered.

Did you try what is suggested in https://stackoverflow.com/questions/62325286/run-github-actions-when-pull-requests-have-a-specific-label ?

@eriknordmark
Copy link
Contributor

yetus failed but generated no annotations against the diffs. Sigh.
Looking at the yetus logs I see this

-1 overall
  
  | Vote |      Subsystem |  Runtime   | Comment
  ============================================================================
  +---------------------------------------------------------------------------
  |      |                |            | Prechecks 
  +---------------------------------------------------------------------------
  |  +1  |       dupname  |   0m  0s   | No case conflicting files found. 
  +---------------------------------------------------------------------------
  |      |                |            | master Compile Tests 
  +---------------------------------------------------------------------------
  +---------------------------------------------------------------------------
  |      |                |            | Patch Compile Tests 
  +---------------------------------------------------------------------------
  |  +1  |     codespell  |   0m  2s   | No new issues. 
  |  +1  |    detsecrets  |  12m 44s   | No new issues. 
  |  +1  |        blanks  |   0m  0s   | The patch has no blanks issues. 
  |  -1  |      yamllint  |   0m  0s   | The patch generated 1 new + 0 unchanged 
  |      |                |            | - 0 fixed = 1 total (was 0)
  +---------------------------------------------------------------------------
  |      |                |            | Other Tests 
  +---------------------------------------------------------------------------
  |      |                |  12m 51s   |

Can you try yamllint manually? Might need to add the yamllint package to your laptop.

@yash-zededa
Copy link
Collaborator Author

We attempted to use https://github.com/marketplace/actions/label-approved-pull-requests, but it did not work properly. Although the PR was marked test-eden, the github actions were not triggered.

Did you try what is suggested in https://stackoverflow.com/questions/62325286/run-github-actions-when-pull-requests-have-a-specific-label ?

Yes, I tried this but unfortunately it didn't worked, The label was added but the github actions was not triggered when the PR was labelled.

@yash-zededa
Copy link
Collaborator Author

yetus failed but generated no annotations against the diffs. Sigh. Looking at the yetus logs I see this

-1 overall
  
  | Vote |      Subsystem |  Runtime   | Comment
  ============================================================================
  +---------------------------------------------------------------------------
  |      |                |            | Prechecks 
  +---------------------------------------------------------------------------
  |  +1  |       dupname  |   0m  0s   | No case conflicting files found. 
  +---------------------------------------------------------------------------
  |      |                |            | master Compile Tests 
  +---------------------------------------------------------------------------
  +---------------------------------------------------------------------------
  |      |                |            | Patch Compile Tests 
  +---------------------------------------------------------------------------
  |  +1  |     codespell  |   0m  2s   | No new issues. 
  |  +1  |    detsecrets  |  12m 44s   | No new issues. 
  |  +1  |        blanks  |   0m  0s   | The patch has no blanks issues. 
  |  -1  |      yamllint  |   0m  0s   | The patch generated 1 new + 0 unchanged 
  |      |                |            | - 0 fixed = 1 total (was 0)
  +---------------------------------------------------------------------------
  |      |                |            | Other Tests 
  +---------------------------------------------------------------------------
  |      |                |  12m 51s   |

Can you try yamllint manually? Might need to add the yamllint package to your laptop.

It might because I removed the trigger for eden.yml and not added the # yamllint disable-line rule:truthy line. Let me add this line.

@yash-zededa
Copy link
Collaborator Author

yash-zededa commented Sep 27, 2023

Yetus checks are passing now. I am waiting for Eden PR-884 to be merged and a tag created so that I can point to a release rather than to a branch.

@eriknordmark
Copy link
Contributor

Yetus checks are passing now. I am waiting for Eden PR-884 to be merged and a tag created so that I can point to a release rather than to a branch.

I've metged 884 in eden

@uncleDecart
Copy link
Member

BuildJet runners are working link to actions

But this PR needs changes: we are still failing on Upgrade test, because we are changing registry in our workflows here and for PRs we set it as evebuild/danger, therefore when upgrading we try to pull from evebuild/danger, it fails.

There are two possible solutions:

  1. We build eve and use docker tag to link evedanger image to lf-edge/eve image and use lf-edge/eve
  2. We add parameter registry parameter to eden utils download command and specify it in our tests, that way no matter where we are running our new workflows (be it on lf-edge/eve repository or any other fork) we will eve from lf-edge for upgrade test

We will try out 1) approach on local fork since it's easier to implement to confirm that this is the problem

@eriknordmark
Copy link
Contributor

But this PR needs changes: we are still failing on Upgrade test, because we are changing registry in our workflows here and for PRs we set it as evebuild/danger, therefore when upgrading we try to pull from evebuild/danger, it fails.

Is the issue that one of the images (the PR) is in evebuild/danger and the other image is in eve/?
If so, how did we handle this in the old tests?

FWIW pollution eve/ with PRs seems bad; less of an issue to insert/link the eve images we will need for the update test in evebuild/danger.

@uncleDecart
Copy link
Member

uncleDecart commented Sep 28, 2023

But this PR needs changes: we are still failing on Upgrade test, because we are changing registry in our workflows here and for PRs we set it as evebuild/danger, therefore when upgrading we try to pull from evebuild/danger, it fails.

Is the issue that one of the images (the PR) is in evebuild/danger and the other image is in eve/?
If so, how did we handle this in the old tests?
FWIW pollution eve/ with PRs seems bad; less of an issue to insert/link the eve images we will need for the update test in evebuild/danger.

Yes it is that issue. We used docker tag to link images we need from evebuild/danger to eve locally, so when eden command runs it'd actually pull evebuild/danger when trying to access image from lf-edge/eve. That Way there was no pollution of lf-edge/eve with PRs and that's why I thought it'd be a good idea to do that.
However, with new workflows it's not that simple. Since we are using reusable workflows and we can't assume that building eve and running docker tag and testing (i.e. executing reusable workflow) will happen on same machine.

We, of course can use artifacts to to push evebulid/danger and I think that will be same polluting as pushing images.

Another alternative which we could explore is GitHub or BuildJet cache, which actually provides 10Gbs of storage (20Gbs f
or BuildJet) for free. That would make it possible to communicate artifacts between jobs without saving them. I think that using cache would actually make sense to speed up our testing even more (i.e. caching kernel builds).

But for now what we should do in my opinion is add parameter registry parameter to eden utils download command and specify it in our tests, that way no matter where we are running our new workflows (be it on lf-edge/eve repository or any other fork) we will eve from lf-edge for upgrade test

Edit: I created PR for eden (link)

@uncleDecart
Copy link
Member

@yash-zededa let's try to run new workflows manually, upgrade test should work now I merged lf-edge/eden#893

@yash-zededa
Copy link
Collaborator Author

@yash-zededa let's try to run new workflows manually, upgrade test should work now I merged lf-edge/eden#893

Thanks @uncleDecart, triggering the workflow manually.

@milan-zededa
Copy link
Contributor

@yash-zededa eden 0.9.0 is here https://github.com/lf-edge/eden/releases/tag/0.9.0

It would be good though to include this PR, otherwise we will not be able to find out why onboarding sometimes fails.

@uncleDecart
Copy link
Member

@milan-zededa merged your PR and released 0.9.1 https://github.com/lf-edge/eden/releases/tag/0.9.1

@yash-zededa it might be a good idea to point EVE to latest eden version if that's possible

@yash-zededa
Copy link
Collaborator Author

@milan-zededa merged your PR and released 0.9.1 https://github.com/lf-edge/eden/releases/tag/0.9.1

@yash-zededa it might be a good idea to point EVE to latest eden version if that's possible

@uncleDecart pointed lf-edge to use 0.9.1 i.e the latest version of eve.

@milan-zededa
Copy link
Contributor

@uncleDecart @yash-zededa It seems that there is some problem with publish-logs/action.yaml not including eve-info.tar.gz in artifacts.
From workflow logs I can see:

Received eve-info.tar.gz with size 1418640
...
Run actions/upload-artifact@v3
  with:
    name: eden-report-networking.tests.txt-tpm-true-ext4
    path: /home/runner/actions-runner/_work/eve/eve/eden/eve-info.tar.gz # created by collect-info action
  /home/runner/actions-runner/_work/eve/eve/eden/trace.log
  /home/runner/actions-runner/_work/eve/eve/eden/info.log
  /home/runner/actions-runner/_work/eve/eve/eden/metric.log
  /home/runner/actions-runner/_work/eve/eve/eden/netstat.log
  /home/runner/actions-runner/_work/eve/eve/eden/console.log
  /home/runner/actions-runner/_work/eve/eve/eden/adam.log

But collected artifact does not contain eve-info.tar.gz.
Maybe working directories between collect/upload actions do not match?

@uncleDecart
Copy link
Member

@milan-zededa created PR, this should fix missing artifacts

Copy link
Contributor

@eriknordmark eriknordmark left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like some commits lost their DCO signoff:
Commit sha: 4eae750, Author: yash-zededa, Committer: yash-zededa; The sign-off is missing.
Commit sha: 9500d5f, Author: yash-zededa, Committer: GitHub; The sign-off is missing.

Also, we are waiting on @uncleDecart @giggsoff to figure out the eden PR.

@uncleDecart
Copy link
Member

@yash-zededa PR is merged, can you run locally and see the results?

@yash-zededa
Copy link
Collaborator Author

@yash-zededa PR is merged, can you run locally and see the results?

@uncleDecart I have executed the action pointing to my forked-repo

https://github.com/yash-zededa/eve/actions/runs/6453056486/job/17515985025

Once the execution is successful and the zip file is part of the artefact we are good to merge this.

@yash-zededa
Copy link
Collaborator Author

Looks like some commits lost their DCO signoff: Commit sha: 4eae750, Author: yash-zededa, Committer: yash-zededa; The sign-off is missing. Commit sha: 9500d5f, Author: yash-zededa, Committer: GitHub; The sign-off is missing.

Also, we are waiting on @uncleDecart @giggsoff to figure out the eden PR.

commits are squashed and signed now.


# yamllint disable-line rule:truthy
on:
workflow_dispatch:
Copy link
Contributor

@giggsoff giggsoff Oct 9, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will dispatch work if we take into account branches below? I can see:

  • if: github.event.review.state == 'approved'
  • if: github.ref == 'refs/heads/master'
  • if: startsWith(github.ref, 'refs/tags')

Do we expect any of them to be true during workflow_dispatch? Do we plan to test only head of master in that case?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am facing weird issue where the GHA are completed within sec. https://github.com/yash-zededa/eve/actions/runs/6453036247

The above actions should have initiated from the start but it got completed in like 8 sec not sure what is wrong. Trying to investigate

name: Eden
# yamllint disable-line rule:comments, rule:truthy
on:
workflow_dispatch
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Did you check that this workflow works in case of manual workflow_dispatch run?
I am not sure that we will pass the check if: ${{ github.event.review.state == 'approved' || github.ref == 'refs/heads/master' || startsWith(github.ref, 'refs/tags') }}. It looks like we will be able to test only the head of master.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if: ${{ github.event.review.state == 'approved' || github.ref == 'refs/heads/master' || startsWith(github.ref, 'refs/tags') }}

The plan is to test the WF on master. I have added the workflow_dispatch only for testing needs and will remove it soon.

${{ github.workspace }}/adam.log
test_suite_pr:
if: github.event.review.state == 'approved'
uses: yash-zededa/eden/.github/workflows/test.yml@master
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would be great to check, prepare new tag in lfedge/eden and point onto it.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, we will. I am just testing the WF of eden for the logs.

@giggsoff
Copy link
Contributor

giggsoff commented Oct 9, 2023

@yash-zededa PR is merged, can you run locally and see the results?

@uncleDecart I have executed the action pointing to my forked-repo

https://github.com/yash-zededa/eve/actions/runs/6453056486/job/17515985025

Once the execution is successful and the zip file is part of the artefact we are good to merge this.

Do you have access to buildjet-4vcpu-ubuntu-2204 runners in your fork? It was my point and it was resolved as I remember.

@yash-zededa
Copy link
Collaborator Author

@yash-zededa PR is merged, can you run locally and see the results?

@uncleDecart I have executed the action pointing to my forked-repo
https://github.com/yash-zededa/eve/actions/runs/6453056486/job/17515985025
Once the execution is successful and the zip file is part of the artefact we are good to merge this.

Do you have access to buildjet-4vcpu-ubuntu-2204 runners in your fork? It was my point and it was resolved as I remember.

I think I had seen a run from my work using the buildjet-4vcpu-ubuntu-2204, let me look for that run

@yash-zededa
Copy link
Collaborator Author

@uncleDecart I think we should merge this PR now. It has been going on since quite sometime :). I have pushed the changes considering we would release 0.9.2 tag for eden, Can you please create the required tag on eden.

@uncleDecart
Copy link
Member

sure @yash-zededa, but does it work with master on lf-edge's eden repo?

Signed-off-by: yash-zededa <[email protected]>
@yash-zededa
Copy link
Collaborator Author

sure @yash-zededa, but does it work with master on lf-edge's eden repo?

Yes, it is working with lf-edge/eve pointing to eden master.

https://github.com/lf-edge/eve/actions/runs/6454241589/job/17519368579

Seems like @giggsoff was correct my forked repo is not having access to the buildjet runners. May be I was confused as we are testing too much, lol :)

@giggsoff
Copy link
Contributor

giggsoff commented Oct 9, 2023

sure @yash-zededa, but does it work with master on lf-edge's eden repo?

Yes, it is working with lf-edge/eve pointing to eden master.

https://github.com/lf-edge/eve/actions/runs/6454241589/job/17519368579

Seems like @giggsoff was correct my forked repo is not having access to the buildjet runners. May be I was confused as we are testing too much, lol :)

Seems there is another error: https://github.com/lf-edge/eve/actions/runs/6454241589/job/17520292589#step:3:6919. And another one: https://github.com/lf-edge/eve/actions/runs/6454241589/job/17520292589#step:3:6845 (we skip ssh option because of the problem).

Also I can see not based on an OCI image here. Which indicates that we try to run vnc test (with qcow2 image inside) on the EVE-OS without hardware acceleration, which is not expected (but looks like a separate issue for the PR).

@uncleDecart
Copy link
Member

@giggsoff I think we can merge it as of now and try to fix tests separately (because this workflow is using 10.4.0 EVE and collect-info.sh should be there)

@yash-zededa
Copy link
Collaborator Author

yash-zededa commented Oct 9, 2023

@giggsoff I think we can merge it as of now and try to fix tests separately (because this workflow is using 10.4.0 EVE and collect-info.sh should be there)

@eriknordmark Appreciate if you can please merge this

Copy link
Contributor

@eriknordmark eriknordmark left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@eriknordmark eriknordmark merged commit bd349a3 into lf-edge:master Oct 9, 2023
20 of 22 checks passed
@uncleDecart
Copy link
Member

@giggsoff I created issues in eden addressing your comment

lf-edge/eden#901
lf-edge/eden#902

@giggsoff
Copy link
Contributor

@giggsoff I created issues in eden addressing your comment

lf-edge/eden#901 lf-edge/eden#902

Thank you!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants