Running LLaMA, a ChapGPT-like large language model released by Meta on Android phone locally.
I use antimatter15/alpaca.cpp, which is forked from ggerganov/llama.cpp.
Alpaca is the fine-tuned version of LLaMA which was released by Stanford University. The alpaca.cpp could run on CPU only mode.
1. Environment
- Device:Xiaomi Pocophone F1
- Android 13
- SoC:Qualcomm Snapdragon 845
- RAM: 6GB
2. Compile alpaca.cpp
Please install Termux first. Then choose one methods from below to compile Alpaca.cpp on your device.
Alpaca requires at leasts 4GB of RAM to run. If your device has RAM >= 8GB, you could run Alpaca directly in Termux or proot-distro (proot is slower). Devices with RAM < 8GB are not enough to run Alpaca 7B because there are always processes running in the background on Android OS. Termux may crash immediately on these devices.
Or you could root your phone and setup a chroot environment. Then mount a swapfile to get more RAM on your device.
2.1. Compile Alpaca.app in Termux
- Install these packages
|
|
- Download Android NDK
|
|
- Complile Alpaca.cpp and download the model
|
|
- Run it
|
|
2.2. In Chroot
Install Chroot Ubuntu and log in to Ubuntu.
If your devices has RAM lower than 8GB, it is recommened to mount a SWAP file.
|
|
- Install dependencies, compile the program, and download the model.
|
|
- Run it.
|
|
2.3. In Proot
- Install Proot Debian and log in to Debian.
|
|
- Install dependencies, compile the program, and download the model.
|
|
- Run it.
|
|
3. Usage
Now you can start chattig with Alpaca.
It will takes Alpaca 30 seconds to start answering your questions.
What can you do with Termux? Well, this looks like the answers from Termux official website :)