Skip to content

Commit

Permalink
docs : add cross-compiling for Android
Browse files Browse the repository at this point in the history
  • Loading branch information
amqdn committed Oct 1, 2024
1 parent f5aaf75 commit fc2b435
Showing 1 changed file with 46 additions and 0 deletions.
46 changes: 46 additions & 0 deletions docs/android.md
Original file line number Diff line number Diff line change
Expand Up @@ -32,3 +32,49 @@ To see what it might look like visually, here's an old demo of an interactive se

https://user-images.githubusercontent.com/271616/225014776-1d567049-ad71-4ef2-b050-55b0b3b9274c.mp4

## Cross-compile using Android NDK
It's possible to build `llama.cpp` for Android on your host system via CMake and the Android NDK. If you are interested in this path, ensure you already have an environment prepared to cross-compile programs for Android (i.e., install the Android SDK). Note that, unlike desktop environments, the Android environment ships with a limited set of native libraries, and so only those libraries are available to CMake when building with the Android NDK (see: https://developer.android.com/ndk/guides/stable_apis.)

Once you're ready and have cloned `llama.cpp`, invoke the following in the project directory:

```
$ cmake \
-DCMAKE_TOOLCHAIN_FILE=$ANDROID_NDK/build/cmake/android.toolchain.cmake \
-DANDROID_ABI=arm64-v8a \
-DANDROID_PLATFORM=android-23 \
-DGGML_OPENMP=OFF \
-DGGML_LLAMAFILE=OFF \
-B build-android
```

Notes:
- Android API level 23 is required (see: https://github.com/abetlen/llama-cpp-python/issues/1284)
- Android does not ship with OpenMP
- `llamafile` does not support Android (see: https://github.com/Mozilla-Ocho/llamafile/issues/325)

Feel free to adjust the Android ABI according to your needs. Once the project is configured:

```
$ cmake --build build-android --config Release -j{n}
$ cmake --install build-android --prefix {install-dir} --config Release
```

After installing, go ahead and download the model of your choice to your host system. Then:

```
$ adb shell "mkdir /data/local/tmp/llama.cpp"
$ adb push {install-dir} /data/local/tmp/llama.cpp/
$ adb push {model}.gguf /data/local/tmp/llama.cpp/
$ adb shell
```

In the `adb shell`:

```
$ cd /data/local/tmp/llama.cpp
$ LD_LIBRARY_PATH=lib ./bin/llama-simple -m {model}.gguf -c {context-size} -p "{your-prompt}"
```

That's it!

Be aware that Android will not find the library path `lib` on its own and does not support `RPATH`, so we must specify `LD_LIBRARY_PATH` in order to run the installed executables. Refer to the previous section for information about `context-size` (very important!) and running other `examples`.

0 comments on commit fc2b435

Please sign in to comment.