|
|
@@ -139,5 +139,5 @@ python3 convert-pth-to-ggml.py models/7B/ 1
|
|
|
In general, it seems to work, but I think it fails for unicode character support. Hopefully, someone can help with that
|
|
|
- I don't know yet how much the quantization affects the quality of the generated text
|
|
|
- Probably the token sampling can be improved
|
|
|
-- x86 quantization support [not yet ready](https://github.com/ggerganov/ggml/pull/27). Basically, you want to run this on Apple Silicon
|
|
|
+- x86 quantization support [not yet ready](https://github.com/ggerganov/ggml/pull/27). Basically, you want to run this on Apple Silicon. For now, on Linux and Windows you can use the F16 `ggml-model-f16.bin` model, but it will be much slower.
|
|
|
|
Georgi Gerganov