feat: React Native support

[raymon-io]

Feature Description

Is it possible for node-llama-cpp to support React Native projects?
I know that this project depends on many node standard libraries, but is it possible to make changes and run in RN runtime? Especially now that it supports arm64 Mac and Windows machines, is it possible to run in arm64 non-node environments?
Even just importing node-llama-cpp necessary functions produces error.

On Android it gives error: The package at "node_modules/node-llama-cpp/dist/evaluator/LlamaModel/LlamaModel.js" attempted to import the Node standard library module "process". It failed because the native React runtime does not include the Node standard library.

On React Native web (using expo) it gives the following error
The resource from “http://localhost:8081/node_modules/expo/AppEntry.bundle?platform=web&dev=true&hot=false&lazy=true&transform.engine=hermes&transform.routerRoot=app” was blocked due to MIME type (“application/json”) mismatch
which comes from a <script> tag in html
<script src="/node_modules/expo/AppEntry.bundle?platform=web&dev=true&hot=false&lazy=true&transform.engine=hermes&transform.routerRoot=app" defer></script>>

The Solution

Not asking to make huge changes but this is more like a theoretical question if experimental supporting of RN runtime is possible after removing or replacing node standard libraries.

Considered Alternatives

llama.rn https://github.com/mybigday/llama.rn
web-llm https://github.com/mlc-ai/web-llm
onnxruntime-react-native https://github.com/microsoft/onnxruntime/tree/main/js/react_native

[giladgd]

The focus of this library is mainly on running on node and node-based environments (like Electron, Bun, etc.), but I don't rule out supporting React-native if it doesn't require significant changes and is not a great maintenance burden.
I haven't used React native for a long time now, so I'm not proficient with its latest features, but from what I've seen, it doesn't use a node runtime and instead has its own kind of runtime that's implemented differently.

node-llama-cpp uses node-addon-api for the native side that uses llama.cpp, and from what I've gathered it's not supported in React native.
If you can find a solution to use a native code that's written using node-addon-api in React native, then it'd be very helpful for adapting node-llama-cpp for React native.
If the process module is the only remaining obstacle, I can refactor the code to use a custom implementation that uses a different abstraction layer when running in React native, but I think the usage of node-addon-api is the main issue.

[raymon-io]

We probably have to wait for this PR facebook/hermes#1377 which is available in RN 0.75.0-rc.5 or newer. But even after that I don't know if android, ios is supported out of the box or not. However RN windows seems to have this implemented already microsoft/hermes-windows#61 . But I have not experimented with any one of them yet. I will continue to research on it and share my findings here. Here is two other discussions to follow for this topic 1) nodejs/abi-stable-node#459 and 2) facebook/hermes#1074

[giladgd]

@raymon-io That looks promising!
Let me know if you need my help to get a POC working when you get to it.

It would be great if you can, as part of your research, find some way to create a basic testsuite for the compatibility of a future support for React Native for Android/iOS/Windows/etc.
This will be important to make sure that changes made to node-llama-cpp for node-based platforms won't break React Native support.