Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Zos light #1

Merged
merged 37 commits into from
Sep 5, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
37 commits
Select commit Hold shift + click to select a range
fecd21a
docs for network light
muhamadazmy Jul 10, 2024
6e9e369
WIP networkd light
ashraffouda Jul 23, 2024
f50626b
Merge branch 'netlight' into zos-light
ashraffouda Jul 23, 2024
b6bb416
wip: testing initial setup
muhamadazmy Jul 23, 2024
6844783
wip: first working network resource
muhamadazmy Jul 23, 2024
ae9081f
finalize Resource creation
muhamadazmy Jul 24, 2024
ef6e2d9
improve nft nat rulese table
muhamadazmy Jul 24, 2024
87fae24
update nft rulese for resource
muhamadazmy Jul 24, 2024
eddc126
scratch board
muhamadazmy Jul 24, 2024
2658cf7
Add basic netlight zbus interface
AbdelrahmanElawady Jul 25, 2024
fa153d7
Add vm-light and network-light workloads
AbdelrahmanElawady Jul 31, 2024
d657225
Add zdb-light
AbdelrahmanElawady Aug 1, 2024
e3f7135
Remove some dependencies on old network
AbdelrahmanElawady Aug 1, 2024
88ae10c
replaces networkd with networkdlight
ashraffouda Aug 7, 2024
259f9a0
change bootstrap to include booting nodes in light mode
ashraffouda Aug 11, 2024
87d16d4
adds updates to bootstrap and update worker
ashraffouda Aug 14, 2024
70c90c4
removes unsed stuff
ashraffouda Aug 15, 2024
b25edf4
make passing version=v4 instead of light
ashraffouda Aug 19, 2024
f044c05
changes ruleset to more clear structure
ashraffouda Aug 20, 2024
73b111d
changes workflow to run on zos-light for testing
ashraffouda Aug 20, 2024
e52e564
change publish.yaml
ashraffouda Aug 22, 2024
57d21f9
fixes workflow
ashraffouda Aug 22, 2024
34c60fc
updates workflow to push bins
ashraffouda Aug 22, 2024
1ce448b
removes bootstrap from zos4 keeping it in the original repo
ashraffouda Aug 22, 2024
73e697b
removes bootstrap from workflows
ashraffouda Aug 22, 2024
532fc64
changes macvlan to veth
ashraffouda Aug 22, 2024
0e6c53b
change all macvlans to veth
ashraffouda Aug 26, 2024
0a1c03e
fix to create mycelium namespace
ashraffouda Aug 29, 2024
c2189b5
better naming for peers
ashraffouda Sep 2, 2024
0d7ff2c
wip
Omarabdul3ziz Sep 2, 2024
96a2e26
Adds cloud console
ashraffouda Sep 2, 2024
13b3c23
bins: exclude librt and libresolv from package content
maxux Sep 4, 2024
3efc9e7
bins: auto append arch and z4 (zos-v4) to package name
maxux Sep 4, 2024
32ac96e
bins: exclude libm as well
maxux Sep 4, 2024
9e3f4bb
bins: skip tpm package for now (too many dependencies libs included)
maxux Sep 4, 2024
5150166
readd bootstrap again
ashraffouda Sep 5, 2024
659d85e
bootstrap: autobuild: adding v4 to symlink name
maxux Sep 4, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/bin-package-18.04.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -53,7 +53,7 @@ jobs:
name: ${{ steps.package.outputs.name }}.flist
- name: Tagging
uses: threefoldtech/publish-flist@master
if: success() && (github.ref == 'refs/heads/main' || startsWith(github.ref, 'refs/tags/v'))
if: success() && (github.ref == 'refs/heads/main' || startsWith(github.ref, 'refs/tags/v') || github.ref == 'refs/heads/zos-light')
with:
token: ${{ secrets.token }}
action: tag
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/bin-package.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -60,7 +60,7 @@ jobs:
name: ${{ steps.package.outputs.name }}.flist
- name: Tagging
uses: threefoldtech/publish-flist@master
if: success() && (github.ref == 'refs/heads/main' || startsWith(github.ref, 'refs/tags/v'))
if: success() && (github.ref == 'refs/heads/main' || startsWith(github.ref, 'refs/tags/v') || github.ref == 'refs/heads/zos-light')
with:
token: ${{ secrets.token }}
action: tag
Expand Down
12 changes: 6 additions & 6 deletions .github/workflows/bins.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -89,12 +89,12 @@ jobs:
package: nnc
secrets:
token: ${{ secrets.HUB_JWT }}
tpm:
uses: ./.github/workflows/bin-package-18.04.yaml
with:
package: tpm
secrets:
token: ${{ secrets.HUB_JWT }}
# tpm:
# uses: ./.github/workflows/bin-package-18.04.yaml
# with:
# package: tpm
# secrets:
# token: ${{ secrets.HUB_JWT }}
qsfs:
uses: ./.github/workflows/bin-package-no-tag.yaml
with:
Expand Down
4 changes: 2 additions & 2 deletions .github/workflows/publish-bootstrap.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -48,7 +48,7 @@ jobs:
user: tf-autobuilder
token: ${{ secrets.HUB_JWT }}
name: ${{ steps.setname.outputs.build }}
target: bootstrap:development.flist
target: bootstrap-v4:development.flist
- name: Symlink flist (release)
if: success() && github.ref == 'refs/heads/main'
uses: threefoldtech/publish-flist@master
Expand All @@ -57,4 +57,4 @@ jobs:
user: tf-autobuilder
token: ${{ secrets.HUB_JWT }}
name: ${{ steps.setname.outputs.build }}
target: bootstrap:latest.flist
target: bootstrap-v4:latest.flist
7 changes: 4 additions & 3 deletions .github/workflows/publish.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -72,7 +72,7 @@ jobs:
# will use the tag value (has to start with v)
- name: Tagging
uses: threefoldtech/publish-flist@master
if: success() && (github.ref == 'refs/heads/main' || startsWith(github.ref, 'refs/tags/v'))
if: success() && (github.ref == 'refs/heads/main' || startsWith(github.ref, 'refs/tags/v') || github.ref == 'refs/heads/zos-light')
with:
token: ${{ secrets.HUB_JWT }}
action: tag
Expand All @@ -82,12 +82,13 @@ jobs:

# only for main branch (devnet)
# this basically releases this build to devnet
# TODO remove zos-light
- name: Cross tagging (development)
if: success() && github.ref == 'refs/heads/main'
if: success() && (github.ref == 'refs/heads/main' || github.ref == 'refs/heads/zos-light')
uses: threefoldtech/publish-flist@master
with:
token: ${{ secrets.HUB_JWT }}
action: crosstag
user: tf-zos
name: development
name: development-v4
target: tf-autobuilder/${{ steps.tag.outputs.reference }}
27 changes: 0 additions & 27 deletions .github/workflows/test-bootstrap.yaml

This file was deleted.

5 changes: 4 additions & 1 deletion bins/bins-extra.sh
Original file line number Diff line number Diff line change
Expand Up @@ -263,13 +263,16 @@ exclude_libs() {
echo "[+] excluding host critical libs"
rm -rf ${ROOTDIR}/usr/lib/ld-linux*
rm -rf ${ROOTDIR}/usr/lib/libc.*
rm -rf ${ROOTDIR}/usr/lib/libm.*
rm -rf ${ROOTDIR}/usr/lib/libdl.*
rm -rf ${ROOTDIR}/usr/lib/librt.*
rm -rf ${ROOTDIR}/usr/lib/libresolv.*
rm -rf ${ROOTDIR}/usr/lib/libpthread.*
}

github_name() {
# force github print
echo "name=${1}" >> $GITHUB_OUTPUT
echo "name=${1}-amd64-z4" >> $GITHUB_OUTPUT
echo "[+] github exported name: ${1}"
}

Expand Down
72 changes: 54 additions & 18 deletions bootstrap/bootstrap/Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

24 changes: 23 additions & 1 deletion bootstrap/bootstrap/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,29 @@ will do a multiple stage bootstrap. Currently this is only two stages:
## How to works

- Bootstrap is used by [0-initramfs](https://github.com/threefoldtech/0-initramfs/blob/development-zos-v3/packages/modules.sh) to basically add `internet` and `bootstrap` services to the base image
- After internet service is fully started, bootstrap will start to download flists needed to for zos node to work properly
- After internet service is fully started, bootstrap will start to download flists needed for zos node to work properly
- As described above bootstrap run in two stages:
- The first stage is used to update bootstrap itself, and it is done like that to avoid re-building the image if we only changed the bootstrap code. this update is basically done from `tf-autobuilder` repo in the [hub/tf-autobuilder](https://hub.grid.tf/tf-autobuilder) and download the latest bootstrap flist
- For the second stage bootstrap will download the flists for that env. bootstrap cares about `runmode` argument that we pass during the start of the node. for example if we passed `runmode=dev` it will get the the tag `development` under [hub/tf-zos](https://hub.grid.tf/tf-zos) each tag is linked to a sub-directory where all flists for this env exists to be downloaded and installed on the node

## Testing in Developer setup

To test bootstrap changes on a local dev-setup you need to do the following

- under zos/qemu `cp -r overlay.normal overlay.custom`
- build `bootstrap` bin
- copy the `bootstrap` bin to overlay.custom/sbin/
- remove dir `overlay.custom/bin`
- remove all files under `overlay.custom/etc/zinit/`
- add the file overlay.custom/etc/zinit/bootstrap.yaml with the following content

```
exec: bootstrap -d
oneshot: true
after:
- internet
```

- remove overlay link under `qemu/overlay `
- create a new link pointing to overlay.custom under zos/qemu `ln -s overlay.custom overlay`
- boot your vm as normal
Loading
Loading