Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: split node from CI #1566

Merged
merged 18 commits into from
Sep 21, 2023
Merged

chore: split node from CI #1566

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
18 commits
Select commit Hold shift + click to select a range
4c9233e
split node from CI
fbarbu15 Sep 12, 2023
1dbb1e5
fix calling the node
fbarbu15 Sep 12, 2023
c3c31b1
remove ref
fbarbu15 Sep 12, 2023
4683d24
make reusable test node workflow
fbarbu15 Sep 12, 2023
27ec075
small fix after ci run
fbarbu15 Sep 12, 2023
a95202f
small fix after ci run
fbarbu15 Sep 12, 2023
7fd6803
small fix after ci run
fbarbu15 Sep 12, 2023
5b9935e
fix debug mismatch
fbarbu15 Sep 12, 2023
ba49fa1
Merge branch 'master' of https://github.com/waku-org/js-waku into cho…
fbarbu15 Sep 13, 2023
8eac299
move node to action folder
fbarbu15 Sep 13, 2023
c577b2c
add ref to the node call
fbarbu15 Sep 13, 2023
442e18c
add sha to the node call
fbarbu15 Sep 13, 2023
a15b027
remove file name from the node call
fbarbu15 Sep 13, 2023
9baf52a
move back node to workflows
fbarbu15 Sep 13, 2023
7a52dd6
move back node to workflows
fbarbu15 Sep 13, 2023
999e95a
fix typo
fbarbu15 Sep 13, 2023
884d376
merge master into branch and resolve conflicts
fbarbu15 Sep 18, 2023
4d54027
Merge branch 'master' into chore/split-ci-into-chunks
weboko Sep 21, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
159 changes: 19 additions & 140 deletions .github/workflows/ci.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,22 +13,13 @@ on:
description: "Docker hub image name taken from https://hub.docker.com/r/statusteam/nim-waku/tags. Format: statusteam/nim-waku:v0.19.0"
required: false
type: string
workflow_call:
inputs:
nim_wakunode_image:
required: false
type: string
caller:
required: false
type: string

env:
NODE_JS: "18"

jobs:
check:
runs-on: ubuntu-latest
if: ${{ inputs.caller == null || inputs.caller != 'nwaku' }}
steps:
- uses: actions/checkout@v3
with:
Expand All @@ -44,7 +35,6 @@ jobs:

proto:
runs-on: ubuntu-latest
if: ${{ inputs.caller == null || inputs.caller != 'nwaku' }}
steps:
- uses: actions/checkout@v3
with:
Expand All @@ -66,7 +56,6 @@ jobs:

browser:
runs-on: ubuntu-latest
if: ${{ inputs.caller == null || inputs.caller != 'nwaku' }}
steps:
- uses: actions/checkout@v3
with:
Expand All @@ -79,145 +68,35 @@ jobs:
- run: npm run test:browser

node:
runs-on: ubuntu-latest
env:
WAKUNODE_IMAGE: ${{ inputs.nim_wakunode_image || 'statusteam/nim-waku:v0.19.0' }}
steps:
- uses: actions/checkout@v3
with:
repository: waku-org/js-waku

- uses: actions/setup-node@v3
with:
node-version: ${{ env.NODE_JS }}

- uses: ./.github/actions/npm

- run: npm run build:esm

- run: npm run test:node
env:
DEBUG: ""

- name: Upload debug logs on failure
uses: actions/upload-artifact@v3
if: failure()
with:
name: debug.log
path: debug.log

- name: Upload nwaku logs on failure
uses: actions/upload-artifact@v3
if: failure()
with:
name: nwaku-logs
path: packages/tests/log/
uses: ./.github/workflows/test-node.yml
with:
nim_wakunode_image: ${{ inputs.nim_wakunode_image || 'statusteam/nim-waku:v0.19.0' }}
test_type: node

node_optional:
runs-on: ubuntu-latest
env:
WAKUNODE_IMAGE: ${{ inputs.nim_wakunode_image || 'statusteam/nim-waku:v0.19.0' }}

steps:
- uses: actions/checkout@v3
with:
repository: waku-org/js-waku

- uses: actions/setup-node@v3
with:
node-version: ${{ env.NODE_JS }}

- uses: ./.github/actions/npm

- run: npm run build:esm

- run: npm run test:optional --workspace=@waku/tests
env:
DEBUG: ""
uses: ./.github/workflows/test-node.yml
with:
nim_wakunode_image: ${{ inputs.nim_wakunode_image || 'statusteam/nim-waku:v0.19.0' }}
test_type: node-optional

node_with_go_waku_master:
runs-on: ubuntu-latest
if: ${{ inputs.caller == null || inputs.caller != 'nwaku' }}
env:
WAKUNODE_IMAGE: "statusteam/go-waku:latest"
WAKU_SERVICE_NODE_PARAMS: "--min-relay-peers-to-publish=0" # Can be removed once https://github.com/status-im/nwaku/issues/1004 is done
DEBUG: "waku*"
steps:
- uses: actions/checkout@v3
with:
repository: waku-org/js-waku

- name: Install NodeJS
uses: actions/setup-node@v3
with:
node-version: ${{ env.NODE_JS }}

- uses: ./.github/actions/npm

- run: npm run build:esm

- run: npm run test:node
env:
DEBUG: "waku:nwaku*,waku:test*"

- name: Upload debug logs on failure
uses: actions/upload-artifact@v3
if: failure()
with:
name: go-waku-debug.log
path: debug.log

- name: Upload logs on failure
uses: actions/upload-artifact@v3
if: failure()
with:
name: go-waku-logs
path: packages/tests/log/
uses: ./.github/workflows/test-node.yml
with:
nim_wakunode_image: statusteam/go-waku:latest
test_type: go-waku-master
debug: waku*

node_with_nwaku_master:
runs-on: ubuntu-latest
if: ${{ inputs.caller == null || inputs.caller != 'nwaku' }}
env:
DEBUG: "waku*"
WAKUNODE_IMAGE: "statusteam/nim-waku:deploy-wakuv2-test"
steps:
- uses: actions/checkout@v3
with:
repository: waku-org/js-waku

- uses: actions/setup-node@v3
with:
node-version: ${{ env.NODE_JS }}

- uses: ./.github/actions/npm

- run: npm run build:esm

- run: npm run test:node
env:
DEBUG: "waku:nwaku*,waku:test*"

- name: Upload debug logs on failure
uses: actions/upload-artifact@v3
if: failure()
with:
name: nwaku-master-debug.log
path: debug.log

- name: Upload logs on failure
uses: actions/upload-artifact@v3
if: failure()
with:
name: nwaku-master-logs
path: packages/tests/log/
uses: ./.github/workflows/test-node.yml
with:
nim_wakunode_image: statusteam/nim-waku:deploy-wakuv2-test
test_type: nwaku-master
debug: waku*

maybe-release:
name: release
runs-on: ubuntu-latest
if: >
github.event_name == 'push' &&
github.ref == 'refs/heads/master' &&
(github.event.inputs.caller == null || github.event.inputs.caller != 'nwaku')
if: github.event_name == 'push' && github.ref == 'refs/heads/master'
needs: [check, proto, browser, node]
steps:
- uses: google-github-actions/release-please-action@v3
Expand Down
54 changes: 54 additions & 0 deletions .github/workflows/test-node.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,54 @@
name: Run Test

on:
workflow_call:
inputs:
nim_wakunode_image:
required: true
type: string
test_type:
required: true
type: string
debug:
required: false
type: string
default: ''

env:
NODE_JS: "18"
WAKU_SERVICE_NODE_PARAMS: ${{ (inputs.test_type == 'go-waku-master') && '--min-relay-peers-to-publish=0' || '' }}
DEBUG: ${{ inputs.debug }}

jobs:
node:
Copy link
Collaborator

@weboko weboko Sep 12, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Github CI works in a way that if is sees a file in the workflows folder then it unfolds into steps in the CI.

One way can be to re-use the action but then you need to place it into actions folder (similar to actions/npm I created there) and from ther you can use it (I am not sure if it is possible to rely on it from different repo)

Or to create a file in workflows and expect it to be run in js-waku and nwaku and have conditions in it to handle that

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tried to move it to actions but unfortunately it doesn't work. Actions is for jobs (like npm) not workflows and I need a workflow for the workflow_call functionality and cross repo usage.
So I need the new workflow to be excluded from the Required status checks in the Branch protection rules and made optional (similar to all other tests).
@weboko do you have access to do this or do I need to ask @fryorcraken ?
Thanks

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@weboko @fryorcraken little help here please

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@fbarbu15 was looking into it yesterday, to me it seemed that some different configuration might help here but if some setting on the repo required - I do not have access.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let me know if you have a better idea on how to handle this.
On the other hand to fix it on my implementation we need something like this(tried it on a forked repo):
image

Copy link
Collaborator

@weboko weboko Sep 19, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In that case we need to ask @fryorcraken to do the setting change

and I think we need to skip node Expected — Waiting for status to be reported in js-waku all together (seems it's already done)

runs-on: ubuntu-latest
env:
WAKUNODE_IMAGE: ${{ inputs.nim_wakunode_image }}
steps:
- uses: actions/checkout@v3
with:
repository: waku-org/js-waku

- uses: actions/setup-node@v3
with:
node-version: ${{ env.NODE_JS }}

- uses: ./.github/actions/npm

- run: npm run build:esm

- run: ${{ (inputs.test_type == 'node-optional') && 'npm run test:optional --workspace=@waku/tests' || 'npm run test:node' }}

- name: Upload debug logs on failure
uses: actions/upload-artifact@v3
if: failure()
with:
name: ${{ inputs.test_type }}-debug.log
path: debug.log

- name: Upload logs on failure
uses: actions/upload-artifact@v3
if: failure()
with:
name: ${{ inputs.test_type }}-logs
path: packages/tests/log/
Loading