Master thesis project - Edge Kubernetes with WebAssembly

Overview

This repository contains resources for my master's thesis focused on optimizing Kubernetes operators for edge orchestration using WebAssembly.
Building on the prototype of Ramlot T. and Van Landuyt K., this project aims to find alternative solutions to wake-up behavior in operators, causing inefficiencies within the WebAssembly-based environment.

Background

This is the third instalment in a series of master's theses, where Kubernetes is adapted for orchestration at the edge by making use of WebAssembly.
It builds on the prior work of Ramlot T. (Github repo), who created the prototype, and Van Landuyt K. (Github repo), who expanded the prototype with predictive capabilities.

During the development of Van Landuyt's solution, problematic behavior was highlighted in the used operators, where the operators would wake-up in a set interval (this through analysis of the Percona MongoDB Operator). This is problematic for the WASM prototype, since unloading is used to minimize the operator footprint. Minimizing the number of wake-ups (i.e. calling the reconcilation function), increases the efficiency of the solution.

"Accepted" reasons to use scheduled reconcilation

From Kubebuilder book: Why not use RequeueAfter X for all scenarios instead of watching resources?:

While RequeueAfter is not the primary method for triggering reconciliations, there are specific cases where it is necessary, such as:

• Observing External Systems: When working with external resources that do not generate events (e.g., external databases or third-party services), RequeueAfter allows the controller to periodically check the status of these resources.
• Time-Based Operations: Some tasks, such as rotating secrets or renewing certificates, must happen at specific intervals. RequeueAfter ensures these operations are performed on schedule, even when no other changes occur.
• Handling Errors or Delays: When managing resources that encounter errors or require time to self-heal, RequeueAfter ensures the controller waits for a specified duration before checking the resource’s status again, avoiding constant reconciliation attempts.

Source problematic behavior

In findings/investigation_percona_mongodb_reconcile, the entire research process can be found, but in the end two main possibilities were found:

1. In order to support sidecar containers
   • These often have the ability to modify the behavior of an application, while not directly modifying the Kubernetes resources, preventing the Reconcile function from being called
   • This can also be seen under the category of "Observing External Systems"
2. In order to manage the secondary resources created when a CR is initialized
   • Doing this properly would involve setting up the correct watches, referencing the CR object and making sure the event filtering (through the use of predicates) is up to snuff

Possible solutions

1. Add CLI command to Kubebuilder (and or operator-sdk) which creates the base for resource watching
   • Both frameworks normally used for project initialization, not during the project
   • Can't be done at initialization since the secondary resources are often added organically during development
   • Would have to add new command / use of framework
2. Modify / expand Kubernetes API + Kubebuilder
   • Register watches to all secondary resources using a single request
   • Has to be flexible -> allow for filtering some resources + use of predicates for each
   • Could for example add its own (optional) Reconcile function (e.g. ReconcileSecondaries), which could have it's own logic
     • Some stuff in the normal Reconcile function, doesn't have to be executed for secondary resources
     • In case an event is missed, a scheduled reconcile of e.g. 30 min should be sufficient (or using the SyncPeriod (in the manager options), since that is its purpose)
   • Could also add the ability to fetch all objects immediately
     • Reconcile always has to fetch the objects -> many GET requests
     • Would go against general Kubernetes behavior, but would still fit in their model of eventual consistency
     • Would have to think about how these objects are then represented (for example, would be handy to have the resources grouped by type or maybe even be able to choose in what format), but this could come from the use of a PoC

Project structure

External resources

• Overleaf: the current version of the master thesis corresponding to this project
• Github: repository of the official WASM operator project

All information is included in this repository. Everything relevant to the prototype will later be upstreamed to the WASM operator project.

Internal folder structure

+-- 📂thesis_resources                  # Resources and documentation specific to the thesis project.
|   +-- 📂findings                      # Detailed research findings and analyses from the project.
|   +-- 📂meeting_notes                 # Summaries from bi-weekly thesis meetings.
|   +-- 📂reconcile_testing_operators   # Modified Kubernetes operators exhibiting undesired reconciliation behavior, with added tracing code and setup.
|   +-- sources.md                      # Table containing overview of most of the used sources.

Note

Meeting notes will often contain duplicate information. It is mostly used for tracking purposes.
The discussed topics are filtered and written down more thoroughly in the other parts of the project.

Planning

Period	Tasks
27/09 - 07/10	Go through previous master theses Research what operators do when activated Investigate how events work on client and server side and why these aren't enough for the WASM-operator
7/10 - 21/10	Analyze Van Landuyt's K.'s operators for the wake-up behavior by investigating the code and traces
21/10 - 4/11	Analyze Van Landuyt's K.'s operators for the wake-up behavior by investigating the code and traces Compare the operator(/controller) architecture from Kubebuilder with kube.rs
4/11 - 18/11	Compare the operator(/controller) architecture from Kubebuilder with kube.rs Investigate how scheduled reconcilation is implemented and why
18/11 - 2/12	Run the WASM-prototype locally and investigate further Find which parts of the Percona MongoDB operator enforce the use of scheduled reconcilation Continue learning Kube.rs
2/12 - 11/12	Work toward prototype solution Work on presentation Create first draft of thesis
11/12 - 19/12	Add automatic watches to PoC Implement feedback presentation Create first draft of thesis

Copyright

This project is released under the Apache License Version 2.0.