Portability

[JohnnySn0w]

If possible, running the entire suite from a docker container would make it highly portable. This seems somewhat infeasible due to the current issue of the disparate gpu/cpu architectural targets for individual components.

A key element may be Cosmopolitan, a universal binary building library. See llamafile for an example of this already done with llama.cpp

[JohnnySn0w]

A caveat to llamafile, is it's running a webserver interface we don't need.