|
|
@@ -10,9 +10,7 @@ Inference of [LLaMA](https://arxiv.org/abs/2302.13971) model in pure C/C++
|
|
|
**Hot topics:**
|
|
|
|
|
|
- [Roadmap (short-term)](https://github.com/ggerganov/llama.cpp/discussions/457)
|
|
|
-- New C-style API is now available: https://github.com/ggerganov/llama.cpp/pull/370
|
|
|
-- Cache input prompts for faster initialization: https://github.com/ggerganov/llama.cpp/issues/64
|
|
|
-- Create a `llama.cpp` logo: https://github.com/ggerganov/llama.cpp/issues/105
|
|
|
+- Support for [GPT4All](https://github.com/ggerganov/llama.cpp#using-gpt4all)
|
|
|
|
|
|
## Description
|
|
|
|
|
|
@@ -37,6 +35,12 @@ Supported platforms:
|
|
|
- [X] Windows (via CMake)
|
|
|
- [X] Docker
|
|
|
|
|
|
+Supported models:
|
|
|
+
|
|
|
+- [X] LLaMA
|
|
|
+- [X] [Alpaca](https://github.com/ggerganov/llama.cpp#instruction-mode-with-alpaca)
|
|
|
+- [X] [GPT4All](https://github.com/ggerganov/llama.cpp#using-gpt4all)
|
|
|
+
|
|
|
---
|
|
|
|
|
|
Here is a typical run using LLaMA-7B:
|
|
|
@@ -222,6 +226,17 @@ cadaver, cauliflower, cabbage (vegetable), catalpa (tree) and Cailleach.
|
|
|
>
|
|
|
```
|
|
|
|
|
|
+### Using [GPT4All](https://github.com/nomic-ai/gpt4all)
|
|
|
+
|
|
|
+- Obtain the `gpt4all-lora-quantized.bin` model
|
|
|
+- It is distributed in the old `ggml` format which is not obsoleted. So you have to convert it to the new format using [./convert-gpt4all-to-ggml.py](./convert-gpt4all-to-ggml.py):
|
|
|
+
|
|
|
+ ```bash
|
|
|
+ python3 convert-gpt4all-to-ggml.py models/gpt4all-7B/gpt4all-lora-quantized.bin ./models/tokenizer.model
|
|
|
+ ```
|
|
|
+
|
|
|
+- You can now use the newly generated `gpt4all-lora-quantized.bin` model in exactly the same way as all other models. The original model is stored in the same folder with a suffix `.orig`
|
|
|
+
|
|
|
### Obtaining and verifying the Facebook LLaMA original model and Stanford Alpaca model data
|
|
|
|
|
|
- **Under no circumstances share IPFS, magnet links, or any other links to model downloads anywhere in this respository, including in issues, discussions or pull requests. They will be immediately deleted.**
|