浏览代码

server : do not get prompt in infill mode (#7286)

* avoid to get prompt in infill mode and embedding mode

* remove embedding mode

* refactor format

---------

Co-authored-by: wudexiang <wudexiang@bytedance.com>
woodx 1 年之前
父节点
当前提交
a5cabd7649
共有 1 个文件被更改,包括 1 次插入1 次删除
  1. 1 1
      examples/server/server.cpp

+ 1 - 1
examples/server/server.cpp

@@ -888,7 +888,7 @@ struct server_context {
         slot.params.input_suffix = json_value(data, "input_suffix", default_params.input_suffix);
 
         // get prompt
-        {
+        if (!task.infill) {
             const auto & prompt = data.find("prompt");
             if (prompt == data.end()) {
                 send_error(task, "Either \"prompt\" or \"messages\" must be provided", ERROR_TYPE_INVALID_REQUEST);