1
0
Эх сурвалжийг харах

llama-run : fix context size (#11094)

Set `n_ctx` equal to `n_batch` in `Opt` class. Now context size is
a more reasonable 2048.

Signed-off-by: Eric Curtin <ecurtin@redhat.com>
Eric Curtin 1 жил өмнө
parent
commit
dc7cef9f37
1 өөрчлөгдсөн 1 нэмэгдсэн , 0 устгасан
  1. 1 0
      examples/run/run.cpp

+ 1 - 0
examples/run/run.cpp

@@ -83,6 +83,7 @@ class Opt {
         }
 
         ctx_params.n_batch        = context_size >= 0 ? context_size : context_size_default;
+        ctx_params.n_ctx          = ctx_params.n_batch;
         model_params.n_gpu_layers = ngl >= 0 ? ngl : ngl_default;
         temperature               = temperature >= 0 ? temperature : temperature_default;