Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

introduce cross-arch testing #28

Open
wants to merge 10 commits into
base: master
Choose a base branch
from

Conversation

jpsollie
Copy link
Contributor

as promised, a cleaned-up version of the cross-compile work of ktest:

  • bcachefs-tools specific code has been eliminated from testrunner
  • cross-compile on host for target has been abandoned (for now) until a better solution comes up
  • various cleanups as suggested

Some user distros do not use standard triplets, but instead, the triplet has another name.
Move the triplet name into a gitignored config file,
and fix a few trivial error messages breaking a cross-compile build:
- use headers-generic instead of headers-$arch
- if ulimit isn't set, don't do it.

Signed-off-by: jpsollie <[email protected]>
@jpsollie
Copy link
Contributor Author

jpsollie commented Nov 18, 2023

some commits were accidentally added a second time, rebased for-review branch

jpsollie and others added 9 commits November 20, 2023 06:17
Allow the tests to run on a non-ABI-compatible cpu,
verify code when the architecture isn't physically available.
in the future, bcachefs-tools should be cross-compiled,
and the list of CPU types should be expanded

Signed-off-by: jpsollie <[email protected]>
Due to bcachefs being mainlined,
it's worth considering a more recent update channel,
which may include bcachefs features in the future.

Necessary adjustments have been made for the packages,
but kernel requirements may still need some extra tweaks

Signed-off-by: jpsollie <[email protected]>

ktest: enable debian sid fallback

RISC-V is currently moving from debian-ports to debian,
in order to protect from unreleased architectures etc,
a fallback scenario is needed:
in case preconfigured settings do not work,
apply a default sid fallback

Signed-Off-By: jpsollie <[email protected]>
Ktest has now been expanded with cross-binary armv7 (hard float).
this should allow some testing on more legacy armv7 hardware,
which may still be present in NAS devices.
However, due to the nature of the ecosystem, a limit is in place:
- 3GB ram for 32 bit systems
- 4 CPUs for 32 bit systems
I think if you need more, you'd better think of going 64-bit.
These constraints allow the ARM emulator to run smoothly.

Signed-off-by:  jpsollie <[email protected]>
Make the rust compiler instsallable via rustup even though it's not in the vm yet.
This avoids rust compile errors on emulated CPUs,
rustup installs for host cpu kernel architecture instead of VM architecture.

Signed-off-by: jpsollie <[email protected]>
Whereas MIPS and MIPS64 are abandoned my manufacturer,
SPARC and POWER seem to be still alive,
and upcoming RISCV also deserves some attention,
so add those to the list of supported architectures.

Signed-off-by: Janpieter Sollie <[email protected]>
This commit fixes a _LOT_ of issues with the previous commits,
but at least there's one supported big endian architecture now,
which is capable of booting and running ktests: s390x

For unsupported architectures, a number of root_create features:
- enabled snapshot lookups for debian archives,
  in case of missing dependencies, it may still be in there
- enabled possibility to build the package from source,
  this is painful
- enabled the possibility during root_create to intervene,
  this is only available on unsupported architectures.

So far, it seems the qemu E6500 emulator for powerPC isn't stable,
and rustup doesn't know about sparc / sparc64,
so both are currently broken.

Re-organised the cross.conf file as the first one was a bit messy.

Signed-Off-by: Janpieter Sollie <[email protected]>
Rustup only has a limited number of architectures,
when the root_image we are creating isn't supported,
try to install the debian rustc compiler.
this will be older, but >95% of time, it should be ok.

Signed-off-by: Janpieter Sollie <[email protected]>
- fix malfunctioning variables
- provide sid fallback for missing architectures
- make the extended installer default for sid

Signed-off-by: Janpieter Sollie <[email protected]>
when running tests, eg test1 on arch 1, and then on arch 2,
the whole bcachefs-tools package needs to be rebuilt:
all binaries of arch 1 need to be erased, and recompiled for arch 2
when going back to arch 1, it happens again

This patch fixes the issue by applying a arch-specific overlay,
so bcachefs-tools doesn't need a full rebuild every time.
The overlay is mounted via fuse-overlay.
If this isn't available, it will fall back to rsync,
causing a larger waste of disk space,
and not-applied-immediately git commits.

Signed-of-by: jpsollie <[email protected]>
@koverstreet koverstreet force-pushed the master branch 5 times, most recently from ef75fd0 to 15af68a Compare January 6, 2024 02:35
@koverstreet koverstreet force-pushed the master branch 3 times, most recently from 8ee3955 to aa1c947 Compare February 1, 2024 04:55
@koverstreet koverstreet force-pushed the master branch 7 times, most recently from da2e4b7 to fc29daf Compare February 17, 2024 03:17
@koverstreet koverstreet force-pushed the master branch 20 times, most recently from feb8eb7 to 55c18fa Compare July 23, 2024 20:59
@koverstreet koverstreet force-pushed the master branch 4 times, most recently from 662d602 to 8216b1f Compare August 8, 2024 20:09
@koverstreet koverstreet force-pushed the master branch 3 times, most recently from c0037b0 to 9a71187 Compare October 14, 2024 21:48
@koverstreet koverstreet force-pushed the master branch 2 times, most recently from 95b4fc3 to caad18f Compare November 8, 2024 00:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant