|
@@ -3,11 +3,10 @@
|
|
|
The purpose of this example is to demonstrate a minimal usage of llama.cpp for running models.
|
|
The purpose of this example is to demonstrate a minimal usage of llama.cpp for running models.
|
|
|
|
|
|
|
|
```bash
|
|
```bash
|
|
|
-llama-run granite-code
|
|
|
|
|
|
|
+llama-run granite3-moe
|
|
|
```
|
|
```
|
|
|
|
|
|
|
|
```bash
|
|
```bash
|
|
|
-llama-run -h
|
|
|
|
|
Description:
|
|
Description:
|
|
|
Runs a llm
|
|
Runs a llm
|
|
|
|
|
|
|
@@ -17,7 +16,7 @@ Usage:
|
|
|
Options:
|
|
Options:
|
|
|
-c, --context-size <value>
|
|
-c, --context-size <value>
|
|
|
Context size (default: 2048)
|
|
Context size (default: 2048)
|
|
|
- -n, --ngl <value>
|
|
|
|
|
|
|
+ -n, -ngl, --ngl <value>
|
|
|
Number of GPU layers (default: 0)
|
|
Number of GPU layers (default: 0)
|
|
|
--temp <value>
|
|
--temp <value>
|
|
|
Temperature (default: 0.8)
|
|
Temperature (default: 0.8)
|
Eric Curtin