|
@@ -310,6 +310,8 @@ Building the program with BLAS support may lead to some performance improvements
|
|
|
```
|
|
```
|
|
|
Note: Because llama.cpp uses multiple CUDA streams for matrix multiplication results [are not guaranteed to be reproducible](https://docs.nvidia.com/cuda/cublas/index.html#results-reproducibility). If you need reproducibility, set `GGML_CUDA_MAX_STREAMS` in the file `ggml-cuda.cu` to 1.
|
|
Note: Because llama.cpp uses multiple CUDA streams for matrix multiplication results [are not guaranteed to be reproducible](https://docs.nvidia.com/cuda/cublas/index.html#results-reproducibility). If you need reproducibility, set `GGML_CUDA_MAX_STREAMS` in the file `ggml-cuda.cu` to 1.
|
|
|
|
|
|
|
|
|
|
+ The environment variable [`CUDA_VISIBLE_DEVICES`](https://docs.nvidia.com/cuda/cuda-c-programming-guide/index.html#env-vars) can be used to specify which GPU(s) will be used.
|
|
|
|
|
+
|
|
|
- **CLBlast**
|
|
- **CLBlast**
|
|
|
|
|
|
|
|
OpenCL acceleration is provided by the matrix multiplication kernels from the [CLBlast](https://github.com/CNugteren/CLBlast) project and custom kernels for ggml that can generate tokens on the GPU.
|
|
OpenCL acceleration is provided by the matrix multiplication kernels from the [CLBlast](https://github.com/CNugteren/CLBlast) project and custom kernels for ggml that can generate tokens on the GPU.
|
|
@@ -348,7 +350,7 @@ Building the program with BLAS support may lead to some performance improvements
|
|
|
cmake --install . --prefix /some/path
|
|
cmake --install . --prefix /some/path
|
|
|
```
|
|
```
|
|
|
|
|
|
|
|
- Where `/some/path` is where the built library will be installed (default is `/usr/loca`l`).
|
|
|
|
|
|
|
+ Where `/some/path` is where the built library will be installed (default is `/usr/local`).
|
|
|
</details>
|
|
</details>
|
|
|
|
|
|
|
|
Building:
|
|
Building:
|
Henri Vasserman