Skip to content

SkiffOS: lightweight & robust cross-compiled Linux distribution optimized for hosting containers.

License

Notifications You must be signed in to change notification settings

jamesoncollins/SkiffOS

 
 

Repository files navigation

Introduction

DOI arXiv

SkiffOS is a lightweight operating system for any Linux-compatible computer, ranging from RPi, Odroid, NVIDIA Jetson, to Desktop PCs, Laptops (i.e. Apple MacBook), Phones (PinePhone), Containers, or Cloud VMs. It is:

  • Adoptable: any userspace can be imported/exported to/from container images.
  • Familiar: uses simple Makefile and KConfig language for configuration.
  • Flexible: supports all major OS distributions inside containers.
  • Portable: containers can be moved between machines of similar CPU type.
  • Reliable: changes inside user environments cannot break the host boot-up.
  • Reproducible: a given Skiff Git tree will always produce identical output.

Uses Buildroot to produce a minimal "single-file" host OS as a standardized base cross-platform operating system "shim" for hosting containers. Most Linux platforms have widely varying requirements for kernel, firmware, and additional hardware support packages. The immutable SkiffOS host system contains everything needed to support the hardware, cleanly separated from the applications.

Getting started

Support Server

The SKIFF_CONFIG comma-separated environment variable selects which configuration layers should be merged together to configure the build.

$ make                             # lists all available layers
$ export SKIFF_CONFIG=pi/4,skiff/core
$ make configure                   # configure the system
$ make compile                     # build the system

After you run make configure Skiff will remember what you selected in SKIFF_CONFIG. The compile command instructs Skiff to build the system.

Adjustments can be made to configuration layers, and make compile can be called again to re-pack the system image without re-building everything.

You can add your SSH public key to the target image by adding it to overrides/root_overlay/etc/skiff/authorized_keys/my-key.pub, or by adding it to your own custom configuration package.

The example above uses pi/4, which can be replaced with any of the hardware support packages listed in the Supported Systems table.

Once the build is complete, it's time to flash the system to a SD card. You will need to switch to sudo bash for this on most systems.

$ sudo bash             # switch to root
$ blkid                 # look for your SD card's device file
$ export PI_SD=/dev/sdz # make sure this is right!
$ make cmd/pi/common/format  # tell skiff to format the device
$ make cmd/pi/common/install # tell skiff to install the os

The device needs to be formatted only one time, after which, the install command can be used to update the SkiffOS images without clearing the persistent state. The persist partition is not touched in this step, so anything you save there, including Docker state and system configuration, will not be modified.

There are many other utility commands made available by Buildroot, which can be listed using make br/help, some examples:

$ make br/menuconfig # optionally explore Buildroot config
$ make br/sdk        # build relocatable SDK for target
$ make br/graph-size # graph the target packages sizes

Connect using SSH to root@my-ip-address to access the SkiffOS system, and connect to core@my-ip-address to access the "Core" system container. The mapping between users and containers can be edited in the /mnt/persist/skiff/core/config.yaml file.

The system can then be upgraded over-the-air (OTA) using the rsync script:

$ ./scripts/push_image.bash root@my-ip-address

The SkiffOS upgrade (or downgrade) will take effect on next reboot.

Supported Systems

CI

SkiffOS is based on Buildroot, which can compile operating systems for any Linux-compatible machine.

Here are the boards/systems currently supported:

Board Config Package Bootloader Kernel Notes
Docker Img virt/docker N/A N/A Run in Docker
Qemu virt/qemu N/A ✔ 5.14.15 Run in QEmu
WSL on Windows virt/wsl N/A ✔ msft-5.4.72 Run in WSL2
Apple Macbook apple/macbook rEFInd ✔ 5.14.15 ✔ Tested
BananaPi M1 bananapi/m1 ✔ U-Boot 2020.10 ✔ 5.14.15 ⚠ Obsolete
BananaPi M1+/Pro bananapi/m1plus ✔ U-Boot 2020.10 ✔ 5.14.15 ⚠ Obsolete
BananaPi M2+ bananapi/m2plus ✔ U-Boot 2020.10 ✔ 5.14.15
BananaPi M3 bananapi/m3 ✔ U-Boot 2020.10 ✔ 5.14.15 ✔ Tested
Intel x86/64 intel/x64 ✔ Grub ✔ 5.14.15 ✔ Tested
NVIDIA Jetson Nano jetson/nano ✔ U-Boot ✔ 4.9.140 ✔ Tested
NVIDIA Jetson TX2 jetson/tx2 ✔ U-Boot ✔ 4.9.140 ✔ Tested
Odroid C2 odroid/c2 ✔ U-Boot 2020.10 ✔ tb-5.13.15 ⚠ Obsolete
Odroid C4 odroid/c4 ✔ U-Boot 2021.01 ✔ tb-5.13.15 ✔ Tested
Odroid U odroid/u ✔ U-Boot 2016.03 ✔ tb-5.13.15 ⚠ Obsolete
Odroid HC1 odroid/xu ✔ U-Boot 2019.04 ✔ tb-5.13.15 ✔ Tested
Odroid HC2 odroid/xu ✔ U-Boot 2019.04 ✔ tb-5.13.15 ✔ Tested
Odroid XU3 odroid/xu ✔ U-Boot 2019.04 ✔ tb-5.13.15 ⚠ Obsolete
Odroid XU4 odroid/xu ✔ U-Boot 2019.04 ✔ tb-5.13.15 ✔ Tested
OrangePi Lite orangepi/lite ✔ U-Boot 2018.05 ✔ 5.14.15
OrangePi Zero orangepi/zero ✔ U-Boot 2018.07 ✔ 5.14.15
PcDuino 3 pcduino/3 ✔ U-Boot 2019.07 ✔ 5.14.15
PcEngines APU2 pcengines/apu2 ✔ CoreBoot ✔ 5.14.15
Pi 0 pi/0 N/A ✔ rpi-5.10.73 ✔ Tested
Pi 1 pi/1 N/A ✔ rpi-5.10.73
Pi 3 + 1, 2 pi/3 N/A ✔ rpi-5.10.73 ✔ Tested
Pi 4 pi/4 N/A ✔ rpi-5.10.73 ✔ Tested
Pi 4 (32bit mode) pi/4x32 N/A ✔ rpi-5.10.73 ✔ Tested
Pine64 H64 pine64/h64 ✔ U-Boot ✔ pine64-5.8.0 ✔ Tested
PineBook Pro pine64/book ✔ U-Boot (bin) ✔ ayufan-5.13.0 ✔ Tested
PinePhone pine64/phone ✔ U-Boot (bin) ✔ megi-5.14.10 ✔ Tested
RockPro64 pine64/rockpro64 ✔ U-Boot (bin) ✔ ayufan-5.13.0 ✔ Tested
USBArmory Mk2 usbarmory/mk2 ✔ U-Boot 2020.10 ✔ 5.14.15 ✔ Tested

All targets marked "tested" use automated end-to-end testing on real hardware. Targets marked "Obsolete" are discontinued by their manufacturer but still have a corresponding SkiffOS configuration and should still work.

Adding support for a board involves creating a Skiff configuration package for the board, as described above. If you have a device that is not yet supported by SkiffOS, please open an issue.

Skiff Core

View Demo

The Skiff Core subsystem, enabled with the skiff/core layer or by selecting any of the core environment packages, automatically configures mappings between users and containerized environments. It maps incoming SSH sessions accordingly:

  • Configured using a YAML configuration file skiff-core.yaml.
  • The container image is either pulled or built from scratch.
  • systemd and/or other init systems operate as PID 1 inside the container.

This allows virtually any workflow to be migrated to Skiff. The config file structure is flexible, and allows for any number of containers, users, and images to be defined and built.

Environment Presets

Any existing GNU/Linux system with compatibility with the running kernel version can be loaded as a Docker image with the docker import command.

All core configurations work with all target platforms:

Distribution Config Package Notes
Alpine Linux core/alpine OpenRC
Debian Bullseye core/debian
DietPi core/dietpi DietPi applications tool
Gentoo core/gentoo Based on latest stage3
NASA cFS Framework core/nasa_cfs Flight software framework
NASA Fprime Framework core/nasa_fprime Flight software framework
NixOS core/nixos
NixOS for PinePhone core/pinephone_nixos
NixOS with XFCE core/nixos_xfce
Ubuntu skiff/core With minimal desktop GUI
PineBook Manjaro KDE core/pinebook_manjaro_kde KDE Variant
PinePhone KDE Neon core/pinephone_neon Ubuntu-based KDE Neon
PinePhone Manjaro KDE core/pinephone_manjaro_kde KDE Variant
PinePhone Manjaro Lomiri core/pinephone_manjaro_lomiri Lomiri variant
PinePhone Manjaro Phosh core/pinephone_manjaro_phosh Phosh variant
PinePhone UBTouch core/pinephone_ubtouch Ubuntu touch

The default configuration creates a user named "core" mapped into a container, but this can be easily configured in the skiff-core.yaml configuration file:

containers:
  core:
    image: skiffos/skiff-core-gentoo:latest
    [...]
users:
  core:
    container: core
    containerUser: core
    [...]

To customize the core configuration after booting into SkiffOS, edit the file at /mnt/persist/skiff/core/config.yaml and run systemctl restart skiff-core to apply.

Release Channels

There are three release channels: next, master, and stable.

Skiff can be upgraded or downgraded (rolled back) independently from the persistent storage partition. This allows for easy OTA, and significant improvements in confidence when upgrading system components.

Configuration Packages/Layers

Skiff supports modular configuration packages. A configuration directory contains kernel configs, buildroot configs, system overlays, etc.

These packages are denoted as namespace/name. For example, an ODROID XU4 configuration would be odroid/xu.

Configuration package directories should have a depth of 2, where the first directory is the category name and the second is the package name.

Package Layout

A configuration package is laid out into the following directories:

├── cflags:         compiler flags in files
├── buildroot:      buildroot configuration fragments
├── buildroot_ext:  buildroot extensions (extra packages)
├── buildroot_patches: extra Buildroot global patches
│   ├── <packagename>: patch files for Buildroot <packagename>
│   └── <packagename>/<version>: patches for package version
├── extensions:     extra commands to add to the build system
│   └── Makefile
├── hooks:          scripts hooking pre/post build steps
│   ├── post.sh
│   └── pre.sh
├── kernel:         kernel configuration fragments
├── kernel_patches: kernel .patch files
├── root_overlay:   root overlay files
├── metadata:       metadata files
│   ├── commands
│   ├── dependencies
│   ├── description
│   └── unlisted
├── resources:     files used by the configuration package
├── scripts:       any scripts used by the extensions
├── uboot:         u-boot configuration fragments
└── uboot_patches: u-boot .patch files

All files are optional.

Out-of-tree configuration packages

You can set the following env variables to control this process:

  • SKIFF_CONFIG_PATH_ODROID_XU: Set the path for the ODROID_XU config package. You can set this to add new packages or override old ones.
  • SKIFF_EXTRA_CONFIGS_PATH: Colon separated list of paths to look for config packages.
  • SKIFF_CONFIG: Name of skiff config to use, or comma separated list to overlay, with the later options taking precedence

These packages will be available in the Skiff system.

Local Overrides

It's often useful to be able to adjust the buildroot, kernel, or other configurations locally during development without actually creating a new configuration layer. This can be easily done with the overrides system.

The overrides directory, as well as the overrides/workspaces/$SKIFF_WORKSPACE directory, are automatically used as additional Skiff configuration packages. You can follow the Skiff configuration package format as defined below to override any of the settings in Buildroot or the Linux kernel, add extra Buildroot packages, add build hooks, etc.

Workspaces

Workspaces allow you to configure and compile multiple systems at a time.

Set SKIFF_WORKSPACE to the name of the workspace you want to use. The Buildroot setup will be constructed in workspaces/$SKIFF_WORKSPACE. You can also place configuration files in overrides/workspaces/$SKIFF_WORKSPACE/ to override settings for that particular workspace locally.

Virtualization

The virt/ packages are designed for running Skiff in various virtualized environments.

Qemu

Here is a minimal working example of running Skiff in Qemu:

$ SKIFF_CONFIG=virt/qemu make configure compile
$ make cmd/virt/qemu/run

Docker

Here is a minimal working example of running Skiff in Docker:

$ SKIFF_CONFIG=virt/docker,skiff/core make configure compile
$ make cmd/virt/docker/buildimage
$ make cmd/virt/docker/run

# inside container
$ su - core

The build command compiles the image, and run executes it.

You can execute a shell inside the container with:

$ make cmd/virt/docker/exec
# alternatively
$ docker exec -it skiff sh

Alternatively, run the latest demo release on Docker Hub:

docker run -t -d --name=skiff \
  --privileged \
  --cap-add=NET_ADMIN \
  --security-opt seccomp=unconfined \
  --stop-signal=SIGRTMIN+3 \
  -v /sys/fs/cgroup:/sys/fs/cgroup:ro \
  -v $(pwd)/skiff-persist:/mnt/persist \
  skiffos/skiffos:latest

Configuration

SkiffOS includes a systemd-based configuration and a standard partition layout, with boot files separated from the persistent data, on default. This can be disabled, overridden, and/or customized by other configuration packages.

NetworkManager

Skiff uses NetworkManager to manage network connections.

Network configurations are loaded from /etc/NetworkManager/system-connections and from skiff/connections on the persist partition.

The configuration file format for these connections is documented here with examples.

You can use nmcli on the device to manage NetworkManager, and any connection definitions written by nmcli device wifi connect or similar will automatically be written to the persist partition and persisted to future boots.

Hostname

You can set the hostname by placing the desired hostname in the skiff/hostname file on the persist partition. You could also set this in one of your config packages by writing the desired hostname to /etc/hostname.

SSH Keys

The system on boot will generate the authorized_keys file for root.

It takes SSH public key files (*.pub) from these locations:

  • /etc/skiff/authorized_keys from inside the image
  • skiff/keys from inside the persist partition

Mount a Disk to a Container

To mount a Linux disk, for example an ext4 partition, to a path inside a Docker container, you can use the Docker Volumes feature:

# Determine the UUID of your volume
# Look for your disk & get the UUID
blkid
# Create "my-volume" with the UUID.
# Replace with your UUID from the previous command.
docker volume create --driver=local --opt type=ext4 --opt device=/dev/disk/by-uuid/d21c6d3a-461e-4d7d-8732-40e56e8f184a my-volume
# Run a container with the drive mounted to a path.
docker run -v my-volume:/my-volume --rm -it alpine sh

The volumes and containers state is stored in the "persist" partition and will remain between reboots, while the SkiffOS host system is ephemeral.

Build in Docker

You can build Skiff inside Docker if you encounter any incompatibility with your build host operating system.

Support

If you encounter issues or questions at any point when using Skiff, please file a GitHub issue and/or Join Discord.

About

SkiffOS: lightweight & robust cross-compiled Linux distribution optimized for hosting containers.

Resources

License

Stars

Watchers

Forks

Packages

No packages published

Languages

  • Shell 55.9%
  • Makefile 16.9%
  • Dockerfile 13.8%
  • C 11.4%
  • Nix 1.3%
  • Batchfile 0.5%
  • Python 0.2%